{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost Parameter Tuning for Rent Listing Inqueries Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rental Listing Inquiries数据集是Kaggle平台上的一个分类竞赛任务，需要根据公寓的特征来预测其受欢迎程度（用户感兴趣程度分为高、中、低三类）。其中房屋的特征x共有14维，响应值y为用户对该公寓的感兴趣程度。评价标准为logloss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4.参数调优（细调）：n_estimators  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>3000</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>750.000000</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>5465</td>\n",
       "      <td>2732.5</td>\n",
       "      <td>1821.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.0</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3275</td>\n",
       "      <td>1637.5</td>\n",
       "      <td>1637.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 228 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.5         3   3000           1200.0      750.000000       -1.5   \n",
       "1        1.0         2   5465           2732.5     1821.666667       -1.0   \n",
       "2        1.0         1   2850           1425.0     1425.000000        0.0   \n",
       "3        1.0         1   3275           1637.5     1637.500000        0.0   \n",
       "4        1.0         4   3350           1675.0      670.000000       -3.0   \n",
       "\n",
       "   room_num  Year  Month  Day       ...        walk  walls  war  washer  \\\n",
       "0       4.5  2016      6   24       ...           0      0    0       0   \n",
       "1       3.0  2016      6   12       ...           0      0    0       0   \n",
       "2       2.0  2016      4   17       ...           0      0    0       0   \n",
       "3       2.0  2016      4   18       ...           0      0    0       0   \n",
       "4       5.0  2016      4   28       ...           0      0    1       0   \n",
       "\n",
       "   water  wheelchair  wifi  windows  work  interest_level  \n",
       "0      0           0     0        0     0               1  \n",
       "1      0           0     0        0     0               2  \n",
       "2      0           0     0        0     0               0  \n",
       "3      0           0     0        0     0               2  \n",
       "4      0           0     0        0     0               2  \n",
       "\n",
       "[5 rows x 228 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# path to where the data lies\n",
    "dpath = './data/'\n",
    "train = pd.read_csv(dpath +\"RentListingInquries_FE_train.csv\")\n",
    "test = pd.read_csv(dpath +\"RentListingInquries_FE_test.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>virtual</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2950</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3758</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3300</td>\n",
       "      <td>1650.000000</td>\n",
       "      <td>1100.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "      <td>4900</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 227 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.0         1   2950      1475.000000     1475.000000        0.0   \n",
       "1        1.0         2   2850      1425.000000      950.000000       -1.0   \n",
       "2        1.0         1   3758      1879.000000     1879.000000        0.0   \n",
       "3        1.0         2   3300      1650.000000     1100.000000       -1.0   \n",
       "4        2.0         2   4900      1633.333333     1633.333333        0.0   \n",
       "\n",
       "   room_num  Year  Month  Day  ...   virtual  walk  walls  war  washer  water  \\\n",
       "0       2.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "1       3.0  2016      6   24  ...         0     0      0    1       0      0   \n",
       "2       2.0  2016      6    3  ...         0     0      0    0       0      0   \n",
       "3       3.0  2016      6   11  ...         0     0      0    0       0      0   \n",
       "4       4.0  2016      4   12  ...         0     0      0    1       0      0   \n",
       "\n",
       "   wheelchair  wifi  windows  work  \n",
       "0           0     0        0     0  \n",
       "1           0     0        0     0  \n",
       "2           0     0        0     0  \n",
       "3           1     0        0     0  \n",
       "4           0     0        0     0  \n",
       "\n",
       "[5 rows x 227 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "发现测试集中没有y，这次作业无法得到在测试集上的表现情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*********** train **********\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 49352 entries, 0 to 49351\n",
      "Columns: 228 entries, bathrooms to interest_level\n",
      "dtypes: float64(9), int64(219)\n",
      "memory usage: 85.8 MB\n",
      "************ test ***********\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 74659 entries, 0 to 74658\n",
      "Columns: 227 entries, bathrooms to work\n",
      "dtypes: float64(9), int64(218)\n",
      "memory usage: 129.3 MB\n"
     ]
    }
   ],
   "source": [
    "print(\"*********** train **********\")\n",
    "train.info()\n",
    "print(\"************ test ***********\")\n",
    "test.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "测试集中少一维，即y标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAELCAYAAAARNxsIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGx5JREFUeJzt3X2wHXWd5/H3h/AgPibIhcokYYKaGUUdo14hypSj6EDAHRMpUChHIkNtxAKF0rUEd5aMPMyM4ygrM8pupogkrkOIiBI1GDMMyII8JGB4CJHlioxcyZJgeNQVivDZP/p35eTm5N7OpU9OTu7nVdV1ur/96z7fwy340v379a9lm4iIiCbs0e0EIiJi95GiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIas2e3E9jZ9t9/f0+fPr3baURE9JTbbrvtEdt9o7Ubd0Vl+vTprFmzpttpRET0FEn/Uaddbn9FRERjOlZUJL1I0q2S7pC0TtLnS/xSSb+QtLYsM0tcki6SNCDpTklvaTnXPEn3lWVeS/ytku4qx1wkSZ36PRERMbpO3v56GjjC9lOS9gJukHR12fcZ21cMa380MKMshwEXA4dJ2g9YAPQDBm6TtNz2o6XNfOBmYAUwG7iaiIjoio5dqbjyVNncqywjzbM/B1hSjrsZmChpMnAUsMr25lJIVgGzy76X277J1fz9S4C5nfo9ERExuo72qUiaIGktsJGqMNxSdl1QbnFdKGmfEpsCPNhy+GCJjRQfbBNvl8d8SWskrdm0adML/l0REdFeR4uK7S22ZwJTgUMlvQE4G3gt8DZgP+CzpXm7/hCPId4uj4W2+2339/WNOiIuIiLGaKeM/rL9GHAdMNv2hnKL62ng68ChpdkgMK3lsKnAQ6PEp7aJR0REl3Ry9FefpIllfV/gvcDPSl8IZaTWXODucshy4KQyCmwW8LjtDcBK4EhJkyRNAo4EVpZ9T0qaVc51EnBVp35PRESMrpOjvyYDiyVNoCpey2x/X9K/S+qjun21Fji1tF8BHAMMAL8FTgawvVnSecDq0u5c25vL+seBS4F9qUZ9ZeRXREQXqRo4NX709/c7T9RH7NoO/6fDu53Cbu/GT9y4Q+0l3Wa7f7R2eaI+IiIak6ISERGNSVGJiIjGpKhERERjUlQiIqIxKSoREdGYFJWIiGhMikpERDQmRSUiIhqTohIREY1JUYmIiMakqERERGNSVCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMR0rKpJeJOlWSXdIWifp8yV+sKRbJN0n6XJJe5f4PmV7oOyf3nKus0v8XklHtcRnl9iApLM69VsiIqKeTl6pPA0cYftNwExgtqRZwBeAC23PAB4FTintTwEetf0a4MLSDkmHACcArwdmA1+TNEHSBOCrwNHAIcCJpW1ERHRJx4qKK0+Vzb3KYuAI4IoSXwzMLetzyjZl/3skqcSX2n7a9i+AAeDQsgzYvt/2M8DS0jYiIrqko30q5YpiLbARWAX8HHjM9rOlySAwpaxPAR4EKPsfB17ZGh92zPbiERHRJR0tKra32J4JTKW6snhdu2blU9vZt6PxbUiaL2mNpDWbNm0aPfGIiBiTnTL6y/ZjwHXALGCipD3LrqnAQ2V9EJgGUPa/AtjcGh92zPbi7b5/oe1+2/19fX1N/KSIiGijk6O/+iRNLOv7Au8F1gPXAseVZvOAq8r68rJN2f/vtl3iJ5TRYQcDM4BbgdXAjDKabG+qzvzlnfo9ERExuj1HbzJmk4HFZZTWHsAy29+XdA+wVNL5wE+BS0r7S4BvSBqgukI5AcD2OknLgHuAZ4HTbG8BkHQ6sBKYACyyva6DvyciIkbRsaJi+07gzW3i91P1rwyP/w44fjvnugC4oE18BbDiBScbERGNyBP1ERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGpKhERERjUlQiIqIxKSoREdGYFJWIiGjMqEVF0ksk7VHW/0jS+yXt1fnUIiKi19S5UrkeeJGkKcA1wMnApZ1MKiIielOdoiLbvwWOBf7J9geo3gkfERGxlVpFRdLbgQ8DPyixTk6ZHxERPapOUTkTOBv4Tnm3yauoXrQVERGxlVGvOGz/GPixpJeU7fuBT3Y6sYiI6D11Rn+9vbytcX3ZfpOkr3U8s4iI6Dl1bn/9d+Ao4NcAtu8A3tnJpCIiojfVevjR9oPDQls6kEtERPS4OqO4HpT0DsCS9qbqT1nf2bQiIqIX1blSORU4DZgCDAIzy3ZERMRWRi0qth+x/WHbB9o+wPZf2v71aMdJmibpWknrJa2TdEaJ/42kX0laW5ZjWo45W9KApHslHdUSn11iA5LOaokfLOkWSfdJurxcSUVERJfUGf21WNLElu1JkhbVOPezwKdtvw6YBZwmaehJ/AttzyzLinLeQ4ATgNcDs4GvSZogaQLwVeBoqif5T2w5zxfKuWYAjwKn1MgrIiI6pM7trz+x/djQhu1HgTePdpDtDbZvL+tPUvXDTBnhkDnAUttP2/4FMAAcWpYB2/fbfgZYCsyRJOAI4Ipy/GJgbo3fExERHVKnqOwhadLQhqT92MFpWiRNpypEt5TQ6ZLulLSo5dxTgNZRZoMltr34K4HHbD87LB4REV1Sp6h8CfiJpPMknQf8BPiHul8g6aXAt4EzbT8BXAy8mqrDf0M5P4DaHO4xxNvlMF/SGklrNm3aVDf1iIjYQXU66pcAxwEPAxuBY21/o87Jy3tXvg180/aV5XwP295i+zngX6hub0F1pTGt5fCpwEMjxB8BJkrac1i83W9YaLvfdn9fX1+d1CMiYgzqvvnxZ8CVwFXAU5IOGu2A0udxCbDe9pdb4pNbmn0AuLusLwdOkLSPpIOBGcCtwGpgRhnptTdVZ/5y26aa2PK4cvy8kl9ERHTJqH0jkj4BLKC6UtlCddvJwJ+McujhwEeAuyStLbHPUY3emlnO8QDwMYAyA/Iy4B6qkWOn2d5ScjgdWAlMABbZXlfO91lgqaTzgZ9SFbGIiOiSOh3uZwB/XOfZlFa2b6B9v8eKEY65ALigTXxFu+PKjMmHDo9HRER31Ln99SDweKcTiYiI3lfnSuV+4DpJPwCeHgq29pNERERAvaLyy7LsXZaIiIi26rz58fMAkl5i+zedTykiInpV3vwYERGNyZsfIyKiMXnzY0RENCZvfoyIiMbkzY8REdGYEa9UyguyPmL7wzspn4iI6GEjXqmUubfm7KRcIiKix9XpU7lR0j8DlwO/f05l6K2OERERQ+oUlXeUz3NbYqZ6lW9ERMTvjdansgdwse1lOymfiIjoYaP1qTwHnL6TcomIiB5XZ0jxKkn/RdI0SfsNLR3PLCIiek6dPpW/Kp+tz6YYeFXz6URERC+rM0vxwTsjkYiI6H113lF/Uru47SXNpxMREb2szu2vt7Wsvwh4D3A7kKISERFbqXP76xOt25JeAXyjYxlFRETPqjX1/TC/BWaM1qiMFrtW0npJ6ySdUeL7SVol6b7yOanEJekiSQOS7pT0lpZzzSvt75M0ryX+Vkl3lWMukqQx/J6IiGhInTc/fk/S8rJ8H7gXuKrGuZ8FPm37dcAs4DRJhwBnAdfYngFcU7YBjqYqVjOA+cDF5fv3AxYAhwGHAguGClFpM7/luNk18oqIiA6p06fyjy3rzwL/YXtwtINsbwA2lPUnJa2nmj5/DvCu0mwxcB3w2RJfYtvAzZImSppc2q6yvRlA0ipgtqTrgJfbvqnElwBzgatr/KaIiOiAOkXll8AG278DkLSvpOm2H6j7JZKmA28GbgEOLAUH2xskHVCaTQFa3zA5WGIjxQfbxCMiokvq9Kl8C3iuZXtLidUi6aXAt4EzbT8xUtM2MY8h3i6H+ZLWSFqzadOm0VKOiIgxqlNU9rT9zNBGWd+7zskl7UVVUL5p+8oSfrjc1qJ8bizxQWBay+FTgYdGiU9tE9+G7YW2+2339/X11Uk9IiLGoE5R2STp/UMbkuYAj4x2UBmJdQmw3vaXW3YtB4ZGcM3j+U7/5cBJZRTYLODxcptsJXCkpEmlg/5IYGXZ96SkWeW7TqLeAIKIiOiQOn0qpwLfLC/qguoKoe1T9sMcDnwEuEvS2hL7HPD3wDJJp1D11xxf9q0AjgEGqIYtnwxge7Ok84DVpd25Q532wMeBS4F9qTro00kfEdFFdR5+/Dkwq/SNyPaTdU5s+wba93tA9VT+8PZm60krW/ctAha1ia8B3lAnn4iI6Lw6z6n8raSJtp8qQ4MnSTp/ZyQXERG9pU6fytG2HxvasP0o1W2qiIiIrdQpKhMk7TO0IWlfYJ8R2kdExDhVp6P+fwHXSPo61XMgf0X1JHxERMRW6nTU/4OkO4H3ltB5tld2Nq2IiOhFda5UAH4K7EV1pfLTzqUTERG9rM7orw8CtwLHAR8EbpF0XKcTi4iI3lPnSuW/Am+zvRFAUh/wb8AVnUwsIiJ6T53RX3sMFZTi1zWPi4iIcabOlcoPJa0ELivbH6KaUiUiImIrdUZ/fUbSscCfUk27stD2dzqeWURE9Jxao7/KtPVXjtowIiLGtfSNREREY1JUIiKiMdstKpKuKZ9f2HnpRERELxupT2WypD8D3i9pKcPejWL79o5mFhERPWekonIOcBbVu9+/PGyfgSM6lVRERPSm7RYV21cAV0j6b7bP24k5RUREj6rznMp5kt4PvLOErrP9/c6mFRERvajOhJJ/B5wB3FOWM0osIiJiK3UefnwfMNP2cwCSFlNNf392JxOLiIjeU/c5lYkt66/oRCIREdH76hSVvwN+KunScpVyG/C3ox0kaZGkjZLubon9jaRfSVpblmNa9p0taUDSvZKOaonPLrEBSWe1xA+WdIuk+yRdLmnvuj86IiI6Y9SiYvsyYBbV3F9XAm+3vbTGuS8FZreJX2h7ZllWAEg6BDgBeH055muSJkiaAHwVOBo4BDixtAX4QjnXDOBR4JQaOUVERAfVuv1le4Pt5bavsv1/ax5zPbC5Zh5zgKW2n7b9C2AAOLQsA7bvt/0MsBSYI0lUz8kMvShsMTC35ndFRESHdGPur9Ml3Vluj00qsSnAgy1tBktse/FXAo/ZfnZYvC1J8yWtkbRm06ZNTf2OiIgYZmcXlYuBVwMzgQ3Al0pcbdp6DPG2bC+03W+7v6+vb8cyjoiI2kYsKpL2aO1of6FsP2x7Sxme/C9Ut7egutKY1tJ0KvDQCPFHgImS9hwWj4iILhqxqJT/+N8h6aAmvkzS5JbNDwBDBWs5cIKkfSQdDMwAbgVWAzPKSK+9qTrzl9s2cC1wXDl+HnBVEzlGRMTY1Xn4cTKwTtKtwG+GgrbfP9JBki4D3gXsL2kQWAC8S9JMqltVDwAfK+daJ2kZ1RP7zwKn2d5SznM6sBKYACyyva58xWeBpZLOp3oY85I6PzgiIjqnTlH5/FhObPvENuHt/off9gXABW3iK4AVbeL38/zts4iI2AXUmVDyx5L+EJhh+98kvZjqqiEiImIrdSaU/M9Uz4P8zxKaAny3k0lFRERvqjOk+DTgcOAJANv3AQd0MqmIiOhNdYrK0+VpdgDKMN7tPhMSERHjV52i8mNJnwP2lfTnwLeA73U2rYiI6EV1ispZwCbgLqohwCuAv+5kUhER0ZvqjP56rkx5fwvVba97y8OHERERWxm1qEh6H/A/gJ9Tzbl1sKSP2b6608lFRERvqfPw45eAd9seAJD0auAHQIpKRERspU6fysahglLcD2zsUD4REdHDtnulIunYsrpO0gpgGVWfyvFUEz1GRERsZaTbX3/Rsv4w8GdlfRMwadvmEREx3m23qNg+eWcmEhERva/O6K+DgU8A01vbjzb1fUREjD91Rn99l2rK+u8Bz3U2nYiI6GV1isrvbF/U8UwiIqLn1SkqX5G0APgR8PRQ0PbtHcsqIiJ6Up2i8kbgI8ARPH/7y2U7Ypf0y3Pf2O0UxoWDzrmr2ynELqZOUfkA8KrW6e8jIiLaqfNE/R3AxE4nEhERva/OlcqBwM8krWbrPpUMKY6IiK3UKSoLxnJiSYuA/0Q1d9gbSmw/4HKqZ14eAD5o+1FJAr4CHAP8Fvjo0EAASfN4/v0t59teXOJvBS4F9qV6x8sZmZI/IqK7Rr39ZfvH7ZYa574UmD0sdhZwje0ZwDVlG+BoYEZZ5gMXw++L0ALgMOBQYIGkoSliLi5th44b/l0REbGTjVpUJD0p6Ymy/E7SFklPjHac7euBzcPCc4DFZX0xMLclvsSVm4GJkiYDRwGrbG+2/SiwCphd9r3c9k3l6mRJy7kiIqJL6rz58WWt25LmUl01jMWBtjeU826QdECJTwEebGk3WGIjxQfbxNuSNJ/qqoaDDjpojKlHRMRo6oz+2ort79L8Mypq91VjiLdle6Htftv9fX19Y0wxIiJGU2dCyWNbNvcA+hnhP+CjeFjS5HKVMpnnX/Y1CExraTcVeKjE3zUsfl2JT23TPiIiuqjOlcpftCxHAU9S9YGMxXJgXlmfB1zVEj9JlVnA4+U22UrgSEmTSgf9kcDKsu9JSbPKyLGTWs4VERFdUqdPZUzvVZF0GdVVxv6SBqlGcf09sEzSKcAvqd4iCdWQ4GOAAaohxSeX794s6Tyef9PkubaHOv8/zvNDiq8uS0REdNFIrxM+Z4TjbPu8kU5s+8Tt7HpPu5MBp23nPIuARW3ia4A3jJRDRETsXCNdqfymTewlwCnAK4ERi0pERIw/I71O+EtD65JeBpxBdVtqKfCl7R0XERHj14h9KuWJ9k8BH6Z6WPEt5SHEiIiIbYzUp/JF4FhgIfBG20/ttKwiIqInjTSk+NPAH1BN5vhQy1QtT9aZpiUiIsafkfpUdvhp+4iIGN9SOCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGdKWoSHpA0l2S1kpaU2L7SVol6b7yOanEJekiSQOS7pT0lpbzzCvt75M0rxu/JSIintfNK5V3255pu79snwVcY3sGcE3ZBjgamFGW+cDFUBUhYAFwGHAosGCoEEVERHfsSre/5gCLy/piYG5LfIkrNwMTJU0GjgJW2d5s+1FgFTB7ZycdERHP61ZRMfAjSbdJml9iB9reAFA+DyjxKcCDLccOltj24hER0SXbfUd9hx1u+yFJBwCrJP1shLZqE/MI8W1PUBWu+QAHHXTQjuYaERE1deVKxfZD5XMj8B2qPpGHy20tyufG0nwQmNZy+FTgoRHi7b5voe1+2/19fX1N/pSIiGix04uKpJdIetnQOnAkcDewHBgawTUPuKqsLwdOKqPAZgGPl9tjK4EjJU0qHfRHllhERHRJN25/HQh8R9LQ9/+r7R9KWg0sk3QK8Evg+NJ+BXAMMAD8FjgZwPZmSecBq0u7c21v3nk/IyIihtvpRcX2/cCb2sR/DbynTdzAads51yJgUdM5RkTE2OxKQ4ojIqLHpahERERjujWkuCe89TNLup3Cbu+2L57U7RQiokG5UomIiMakqERERGNSVCIiojEpKhER0ZgUlYiIaEyKSkRENCZFJSIiGpOiEhERjUlRiYiIxqSoREREY1JUIiKiMSkqERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGNSVGJiIjGpKhERERjer6oSJot6V5JA5LO6nY+ERHjWU8XFUkTgK8CRwOHACdKOqS7WUVEjF89XVSAQ4EB2/fbfgZYCszpck4REeNWrxeVKcCDLduDJRYREV2wZ7cTeIHUJuZtGknzgfll8ylJ93Y0q+7aH3ik20nUpX+c1+0UdiU99bcDYEG7fwXHrZ76++mTO/y3+8M6jXq9qAwC01q2pwIPDW9keyGwcGcl1U2S1tju73YesePyt+tt+ftVev3212pghqSDJe0NnAAs73JOERHjVk9fqdh+VtLpwEpgArDI9roupxURMW71dFEBsL0CWNHtPHYh4+I2324qf7velr8fIHubfu2IiIgx6fU+lYiI2IWkqOwmMl1N75K0SNJGSXd3O5fYMZKmSbpW0npJ6ySd0e2cui23v3YDZbqa/wP8OdUw69XAibbv6WpiUYukdwJPAUtsv6Hb+UR9kiYDk23fLullwG3A3PH8716uVHYPma6mh9m+Htjc7Txix9neYPv2sv4ksJ5xPqtHisruIdPVRHSZpOnAm4FbuptJd6Wo7B5qTVcTEZ0h6aXAt4EzbT/R7Xy6KUVl91BrupqIaJ6kvagKyjdtX9ntfLotRWX3kOlqIrpAkoBLgPW2v9ztfHYFKSq7AdvPAkPT1awHlmW6mt4h6TLgJuCPJQ1KOqXbOUVthwMfAY6QtLYsx3Q7qW7KkOKIiGhMrlQiIqIxKSoREdGYFJWIiGhMikpERDQmRSUiIhqTohIREY1JUYkAJP2kRpszJb24w3nMHO05B0kflfTPDX9v4+eM8SlFJQKw/Y4azc4EdqiolNcS7IiZwLh+eC56W4pKBCDpqfL5LknXSbpC0s8kfVOVTwJ/AFwr6drS9khJN0m6XdK3yqSCSHpA0jmSbgCOl/RqST+UdJuk/y3ptaXd8ZLulnSHpOvLFDvnAh8qT2Z/qEbefZK+LWl1WQ6XtEfJYWJLuwFJB7Zr3/g/zBjX9ux2AhG7oDcDr6ealPNG4HDbF0n6FPBu249I2h/4a+C9tn8j6bPAp6iKAsDvbP8pgKRrgFNt3yfpMOBrwBHAOcBRtn8laaLtZySdA/TbPr1mrl8BLrR9g6SDgJW2XyfpKuADwNfLdz5g+2FJ/zq8PfC6F/jPK+L3UlQitnWr7UEASWuB6cANw9rMAg4BbqzmFGRvqvm7hlxejn8p8A7gW6UdwD7l80bgUknLgLHObvte4JCWc7+8vIHwcqqi9XWqCUYvH6V9RCNSVCK29XTL+hba/3siYJXtE7dzjt+Uzz2Ax2zPHN7A9qnlKuJ9wFpJ27SpYQ/g7bb/31bJSTcBr5HUB8wFzh+l/Ri+OmJb6VOJqO9JYOj/6m8GDpf0GgBJL5b0R8MPKC9s+oWk40s7SXpTWX+17VtsnwM8QvVOnNbvqONHVDNUU845s3yvge8AX6aalv3XI7WPaEqKSkR9C4GrJV1rexPwUeAySXdSFZnXbue4DwOnSLoDWAfMKfEvSrpL0t3A9cAdwLVUt6dqddQDnwT6Jd0p6R7g1JZ9lwN/yfO3vkZrH/GCZer7iIhoTK5UIiKiMemoj9hFSToZOGNY+Ebbp3Ujn4g6cvsrIiIak9tfERHRmBSViIhoTIpKREQ0JkUlIiIak6ISERGN+f+UQGoL+CmGNwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(train.interest_level);\n",
    "pyplot.xlabel('interest_level');\n",
    "pyplot.ylabel('Number of occurrences');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "总共有三类样本，每类样本分布的并不均匀，一般类别之间的数量差别在十倍以上时才考虑使用class_weight，这里暂不考虑"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop ids and get labels\n",
    "y_train = train['interest_level']\n",
    "\n",
    "train = train.drop([\"interest_level\"], axis=1)\n",
    "X_train = np.array(train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二轮参数调整得到的最优值max_depth = 5 ， min_child_weight = 2，本轮返回来调节n_estimators,除了需要调优的参数和已暂时确认的两个参数，其余参数继续默认值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再次调整弱学习器数目"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#直接调用xgboost内嵌的交叉验证（cv），可对连续的n_estimators参数进行快速交叉验证\n",
    "#而GridSearchCV只能对有限个参数进行交叉验证\n",
    "def modelfit(alg, X_train, y_train, cv_folds=None, early_stopping_rounds=10):\n",
    "    xgb_param = alg.get_xgb_params()\n",
    "    xgb_param['num_class'] = 3\n",
    "    \n",
    "    #直接调用xgboost，而非sklarn的wrapper类\n",
    "    xgtrain = xgb.DMatrix(X_train, label = y_train)\n",
    "        \n",
    "    cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], folds =cv_folds,\n",
    "             metrics='mlogloss', early_stopping_rounds=early_stopping_rounds)\n",
    "  \n",
    "    cvresult.to_csv('2_nestimators.csv', index_label = 'n_estimators')\n",
    "    \n",
    "    #最佳参数n_estimators\n",
    "    n_estimators = cvresult.shape[0]\n",
    "    \n",
    "    # 采用交叉验证得到的最佳参数n_estimators，训练模型\n",
    "    alg.set_params(n_estimators = n_estimators)\n",
    "    alg.fit(X_train, y_train, eval_metric='mlogloss')\n",
    "    \n",
    "    #Predict training set:\n",
    "    train_predprob = alg.predict_proba(X_train)\n",
    "    logloss = log_loss(y_train, train_predprob)\n",
    "        \n",
    "    #Print model report:\n",
    "    print (\"logloss of train :\" )\n",
    "    print (logloss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of train :\n",
      "0.5000332000337464\n"
     ]
    }
   ],
   "source": [
    "xgb2_3 = XGBClassifier(\n",
    "        learning_rate =0.09,\n",
    "        n_estimators=1000,  #虽然知道n_estimator应该在337左右，但设置1000也没关系，会自动返回最优值\n",
    "        max_depth=5,\n",
    "        min_child_weight=2,\n",
    "        gamma=0,\n",
    "        subsample=0.5,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb2_3, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEXCAYAAAC3c9OwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmYXGWZ9/HvXb0l6SVrZyF7Qth3wq7sKIuCICIgKuiY0XdwHUdxeR2GeRF1XBGUQQUUFATGBRW3AVFAWRL2EJaQPWRPOp1O0p3urvv943kqXelU9RK6+lR3/T7Xda6uOufUOXef6q5fPc/ZzN0RERHpLJV0ASIiUpwUECIikpMCQkREclJAiIhITgoIERHJSQEhIiI5KSBEspjZ583sh0nXIVIMFBADjJnVmNkSM7s0a1ytmS0zswuzxs02s9+a2SYzazCzF83sWjMbGadfbmbtZtYUh0Vm9pEC136yma0o5Dp6I1c97v5ld/+nAq1viZmdXohlF0J/vV8DbbuUEgXEAOPuTcAc4DtmVh9Hfw2Y6+73ApjZ8cBDwKPAfu4+AjgTaAMOzVrcP9y9xt1rgAuBr5nZ4f3zm0hvmFl50jVICXJ3DQNwAG4D7gROBjYAE7KmPQJ8t5vXXw480mncE8ClWc/PBeYDDYTA2T9r2v5xXEOc59ysaWcDLwJbgJXAp4FqYDuQBprisFee3+tG4Hfx9Y8DM3uwPfYD/gxsBF4GLtqTeoCrgTvi66YBDlwBLAc2AR8GjgKei7/7DVnrmQk8GN+P9cBPgRFx2u1xXdvjuj7Tg228BPhsXFcLUB6fr4y/y8vAaTm2xbHAaqAsa9z5wHPx8dHAXKARWAN8M882PRlYkWfacOAnwDpgKfBFIBWnlQHfiNtgMXBl3I7leZa1BDg9z7QPAQvj+3pf5m8GMOBbwFpgc9xGB+V7v5P+fx2oQ+IFaNjDNw5GAqviP+EVWeOrgXbg5G5efzlZARE/9BqAfeLzfYCtwBlABfCZ+I9aGZ8vBD4fn58a/xn3ja9dBbw5q84j4uO8HzhZddwWPwyOjh+IPwXu6uY11YQP8Cvia46I2+XA3tZD7oC4CRgCvAVoBn4FjAUmxg+ok+L8e8ftVQXUA38Dvp217F0+CLvaxlnzPwNMBoYC+8bfc6+s+nKGJ/AacEbW83uAq+LjfwDvjY9rgGPzLCPv+0UIh18DtbGOV4APxmkfJnxAT4rb+3/Zg4CIf1fr4/tZBXwX+Fuc9lZgHjCCEBb7E78k5Xu/NfR+UBfTAOXumwjfPIcBv8iaNJLQdbg6M8LMvhb3Q2w1sy9mzXtsHN9EaD3cDrwap70b+J27/9ndW4GvEz6kjid8Q60BvuLuO9z9QeC3wCXxta3AAWZW5+6b3P2pXv56v3D3J9y9jRAQh3Uz/9uAJe5+q7u3xfX9D6HbrC/q+U93b3b3PxE+0O9097XuvhJ4GDgcwN0Xxu3V4u7rgG8CJ3Wx3K62ccb17r7c3bcTgr8q/i4V7r7E3V/Ls+w7ie+HmdUSvlXfmbU99jazMe7e5O6P9WZjmFlZrP1z7r7F3ZcQWgzvjbNcBHzH3VfEv9Ov9Gb5Wd4D3OLuT7l7C/A54DgzmxZ/h1pCy9HcfYG7r8r6/d7I+y2RAmKAMrPLCN/c/hf4atakTYSujAmZEe7+GQ/7IX5J+Iad8Zi7j/CwD2I8cCDw5ThtL0LXQWYZacK314lx2vI4LmNpnAbwTsIH0lIz+6uZHdfLX2911uNthDDqylTgmBh2DWbWQPhwGd9H9azJerw9x/MaADMba2Z3mdlKM2sE7gDGdLHcrrZxxvKs6QuBTxBaOWvjuvbKs+yfAReYWRVwAfCUu2fW9UFC6+UlM3vSzN7WRY25jCG0HJdmjct+//fKrrvT497ovH2aCN13E+OXkhsI3ZFrzOxmM6uLs77R91siBcQAZGZjCf2vHwL+GbjIzE4EcPethH77C3qzTHdfQ/jW/fY46nXCB29mnUbo6lgZp002s+y/nylxGu7+pLufR+iG+RVwd2Y1vampF5YDf41hlxlq3P0j/VzPdXGZh7h7HXAZofsjo/P6utrGOV/j7j9z9zfF1zm7fjnInu9FwofrWcClhMDITHvV3S8hbI+vAveaWXXPf03WE76lT80at/P9J3TxTMqaNrkXy87WeftUA6Pp+Du73t2PJHyx2Qf4tzg+3/stvaSAGJhuAH7l7n+JzerPAD+I3xaJzz9gZlfFMMHMJgHT8y3QzEYTdmTOj6PuBs4xs9PMrAL4V8KO0r8TAmgr8BkzqzCzkwnBcpeZVZrZe8xseOw2aSR0jUD45j3azIb30XbI+C2wj5m9N9ZTYWZHmdn+/VxPLWEHdIOZTSR+YGVZA8zIet7VNt6Nme1rZqfG97mZ0HppzzVv9DPgY8CJhH0QmeVcZmb1scXSEEfnXY6ZDckeCC3Uu4Fr4yHWU4FPEVpMmd/r42Y20cxGEHasd6ei03rKY/1XmNlh8Xf+MvC4uy+J7+8xcbttjdujvZv3W3or6Z0gGno3AO8gfLMa0Wn8A8C1Wc+PAe4nfAA0AC8A1wKj4/TLCf84mSN41hL6qMdmLeN8ws7GzcBfiTt947QD47jNcZ7z4/hK4A+Erq5G4EngTVmvu4XQTdBA/qOY/l/W85PpZsd2nG9fwpFP6+LyHyTsu+hVPeTeSV2eNf8Ksg4AIHwofjFrm8yL2/MZwgf+iqx5zwOWxXV9ugfbeAm77tQ+hLCvaAthR/5vc23DrPmnED7Mf9dp/B3x/W4ifCF4R57Xnxx//87D3oR9XXfE7b0c+BIdRzGVE1q4GwhHMX2S0OKwPOtZkmMd/y9O+zBhh3vm950Ux59GOHKpiY4jxmq6e7819G6wuLFFRArCzM4CbnL3qd3OLEVFXUwi0qfMbKiZnW1m5bGr7d8JB0jIAKMWhAwIZvZm4Pe5pnk4CkuKhJkNI3SX7UfYT/I74OPu3phoYdJrCggREclJXUwiIpLTgLsA2JgxY3zatGlJlyEiMqDMmzdvvbvXdz9nhwEXENOmTWPu3LlJlyEiMqCY2dLu59qVuphERCQnBYSIiOSkgBARkZwUECIikpMCQkREclJAiIhITgoIERHJqWQC4vWNTTz6wmt4Ot39zCIiUjoB8ZvvfZYT7j2Cbdu3JV2KiMiAUDIBcfisKQBsa9yYcCUiIgNDyQRE2bBwP/NtWzYlXImIyMBQMgFRMWwEANubGrqZU0REoIQCorI63Je+RQEhItIjJRMQQ2pGAdC6dXPClYiIDAylExC1oYupfbtaECIiPVEyAVFdNxKA9u26La6ISE+UTkDUhIDwZgWEiEhPlExApCoq2e6V2A4FhIhIT5RMQABstWGkdjQlXYaIyIBQUgGxPVVN+Y4tSZchIjIglFRANKeqqWhTQIiI9ETBAsLMbjGztWb2Qp7pZmbXm9lCM3vOzI4oVC0ZLeU1VLZvLfRqREQGhUK2IG4Dzuxi+lnArDjMAb5fwFoAaC2vYWhaASEi0hMFCwh3/xvQ1aVTzwN+4sFjwAgzm1CoegDaK2oYktblvkVEeiLJfRATgeVZz1fEcbsxszlmNtfM5q5bt26PV5jesoYaVwtCRKQnkgwIyzHOc83o7je7+2x3n11fX7/na5w0mxprprmlZc+XISJSIpIMiBXA5Kznk4DXC7nC1NBwPabGhvWFXI2IyKCQZEDcB7wvHs10LLDZ3VcVcoVl1eFyG1s3byjkakREBoXyQi3YzO4ETgbGmNkK4N+BCgB3vwm4HzgbWAhsA64oVC0ZldXhkt/bGxUQIiLdKVhAuPsl3Ux34F8Ktf5cqmpDQDTrvtQiIt0qqTOph9aOBqC1SQEhItKdkgqI6hFjAGjbtinhSkREil9JBURtDAjXXeVERLpVUgFRVjWMFq+AZgWEiEh3SiogAJqsmrKWzUmXISJS9EouILamaqjQXeVERLpVcgHRXFare0KIiPRAyQXEmtYh6mISEemBkguImhFjGGG6L7WISHdKLiDah4xkuG8hnMgtIiL5lFxAMHQUtbadLdu2J12JiEhRK7mASNWEk+UaN+75jYdEREpByQVERU24HlPTpjUJVyIiUtxKLiCG1IU70m1rWJtwJSIixa3kAmLYiLEA7Niiu8qJiHSl5AKiduQ4AFoVECIiXSq5gKgZGbqYfJvuCSEi0pWSC4hUVTXNVMJ2BYSISFdKLiAAGq2O8mYFhIhIV0oyIDamq2nRPggRkS6VZEAwbDTjUroek4hIV0oyIFqGjKEurftSi4h0pSQDIj2sntHeQPOOtqRLEREpWiUZEFY7jqG2gw2btKNaRCSfkgyIirpwstzm9a8nXImISPEqaECY2Zlm9rKZLTSzq3JMn2pmD5jZc2b2kJlNKmQ9GUNH7QXA1vUr+2N1IiIDUsECwszKgBuBs4ADgEvM7IBOs30d+Im7HwJcA1xXqHqy1Y4JAdHcsLo/ViciMiAVsgVxNLDQ3Re5+w7gLuC8TvMcADwQH/8lx/SCGD4mNFTaNisgRETyKWRATASWZz1fEcdlexZ4Z3x8PlBrZqMLWBMAlXX1tGOwVZf8FhHJp5ABYTnGdb4R9KeBk8zsaeAkYCWw27GnZjbHzOaa2dx16/rgTnCpMjZ6HRvXLO9+XhGRElXIgFgBTM56PgnY5bAhd3/d3S9w98OBL8RxmzsvyN1vdvfZ7j67vr6+T4rbVjmKyZVb+2RZIiKDUSED4klglplNN7NK4GLgvuwZzGyMmWVq+BxwSwHr2cX2yjHUtG3or9WJiAw4BQsId28DrgT+CCwA7nb3+WZ2jZmdG2c7GXjZzF4BxgHXFqqeznYMGc3w9gbcO/d6iYgIQHkhF+7u9wP3dxr3pazH9wL3FrKGfNLDxjKGBra2tFEzpCKJEkREilpJnkkNkKodR5W1sn6DLvstIpJLyQZE5YjxADSuW5FwJSIixalkA2LYqAkAbN24KuFKRESKU8kGRN3ocM5eyyZdsE9EJJfSDYgxISDatuhsahGRXEo2IFLVo2kjhTetSboUEZGiVLIBQSpFQ2oUlVt1wT4RkVxKNyCALZXjqGlRC0JEJJeSDojtwyYwun0d6bTOphYR6aykA8LrJjKBDazf0px0KSIiRaekA6Ji1GSqrJXVq3WynIhIZyUdEMPqpwKwefXihCsRESk+JR0QIybMAGD7+qUJVyIiUnxKOiBq6qcD8MKL8xOuRESk+JR0QDBsFC1UcmBNU9KViIgUndIOCDM2lY9l2DZdsE9EpLPSDghg65Bx1LXqZDkRkc5KPiB21ExknK9na0tb0qWIiBSVkg8IGz6RsTSwamNj0qWIiBSVkg+IqtFTSZmzfpUOdRURyVbyAVE3fhoAP/3To8kWIiJSZEo+IEaODyfLnTp+R8KViIgUl5IPiNSoaaQxbNOipEsRESkq3QaEmc00s6r4+GQz+5iZjSh8af2kvIqN5eOoaVqSdCUiIkWlJy2I/wHazWxv4EfAdOBnBa2qnzVVT2Vs60p2tKWTLkVEpGj0JCDS7t4GnA98290/CUwobFn9q33UDKbZapZt2Jp0KSIiRaMnAdFqZpcA7wd+G8dV9GThZnammb1sZgvN7Koc06eY2V/M7Gkze87Mzu556X2nauw+1Nk2Pnf7g0msXkSkKPUkIK4AjgOudffFZjYduKO7F5lZGXAjcBZwAHCJmR3QabYvAne7++HAxcD3elN8Xxk5eX8A3jNLZ1OLiGSUdzeDu78IfAzAzEYCte7+lR4s+2hgobsviq+9CzgPeDF78UBdfDwceL3npfed6gn7ANC69pUkVi8iUpR6chTTQ2ZWZ2ajgGeBW83smz1Y9kRgedbzFXFctquBy8xsBXA/8NE8Ncwxs7lmNnfdunU9WHUvjZhKG2WUN+jOciIiGT3pYhru7o3ABcCt7n4kcHoPXmc5xnmn55cAt7n7JOBs4HYz260md7/Z3We7++z6+voerLqXysrZVLkXtduW9f2yRUQGqJ4ERLmZTQAuomMndU+sACZnPZ/E7l1IHwTuBnD3fwBDgDG9WEef2VY7lYntK2lsbk1i9SIiRacnAXEN8EfgNXd/0sxmAK/24HVPArPMbLqZVRJ2Qt/XaZ5lwGkAZrY/ISAK0IfUvXlbRjHV1rBwja7qKiICPQgId7/H3Q9x94/E54vc/Z09eF0bcCUhXBYQjlaab2bXmNm5cbZ/BT5kZs8CdwKXu3vnbqh+ccoJxzPMWliyeGESqxcRKTrdHsVkZpOA7wInEPYhPAJ83N1XdPdad7+fsPM5e9yXsh6/GJebuBFTDgRg87IXgGOSLUZEpAj0pIvpVkLX0F6Eo5B+E8cNKjbuoPBz7fyEKxERKQ49CYh6d7/V3dvicBtQgEOJEjZsFI0V9YxsfIX2dCK9XCIiRaUnAbHezC4zs7I4XAZsKHRhSViQnsI+LGWJrskkItKjgPgA4RDX1cAq4ELC5TcGnRkHHc1MW8lLK9cnXYqISOJ6chTTMnc/193r3X2su7+DcNLcoDNi+uFUWjtrFr2QdCkiIonb0zvKfapPqygSFRMOBqB15fMJVyIikrw9DYhcl9EY+MbMos0qGLLppaQrERFJ3J4GxOA8zKesgs01M5jaupiVDduTrkZEJFF5A8LMtphZY45hC+GciEEpNf4g9k8tZe6SjUmXIiKSqLwB4e617l6XY6h1927PwB6o6qYfyThr4JVXdW8IESlte9rFNGiVTQmX2dix5LGEKxERSZYCorPxh9BmlYzb/BxbdOlvESlhCojOyivZOuZgDk+9ytPLGpKuRkQkMQqIHIY2r+UgW8zTi1YnXYqISGJ6ck/qXEczLTezX8abBw06lWd9mSpr48nHHkq6FBGRxPTkaKRvEm4V+jPCCXIXA+OBl4FbgJMLVVxiJh8NwIHtL7O1pY3qqkF70JaISF496WI6093/2923uHuju98MnO3uPwdGFri+ZNSOp7l6EofwMn9/bVBeuFZEpFs9CYi0mV1kZqk4XJQ1bXCeUQ1UTD+eY1Mv8deXtB9CREpTTwLiPcB7gbVxeC9wmZkNJdxzelAq2/tURlsjz8/7OwndJltEJFHddq67+yLg7XkmP9K35RSRGScDcAzPsnBtE7PG1SZajohIf+vJUUyT4hFLa81sjZn9j5lN6o/iElU3gdbyGt6UeoGHXl6XdDUiIv2uJ11MtwL3ES7QNxH4TRw36FXMfh/HpF7ipgfmJ12KiEi/60lA1Lv7re7eFofbgPoC11UcZpxClbWyf+t8lm3YlnQ1IiL9qicBsd7MLjOzsjhcBpTGsZ/TTsAx3px6jkt/qIv3iUhp6UlAfAC4CFgNrAIuBK4oZFFFo7Iam3kq51Y9xbAKXZVEREpLt5967r7M3c9193p3H+vu7wAu6MnCzexMM3vZzBaa2VU5pn/LzJ6JwytmVnxXxzvgPCa0r6J83XwWrGpMuhoRkX6zp1+LP9XdDGZWBtwInAUcAFxiZgdkz+Pun3T3w9z9MOC7wC/2sJ7C2e9tOMY5ZY9x+a1PJF2NiEi/2dOAsB7MczSw0N0XufsO4C7gvC7mvwS4cw/rKZzq0diMk7hw2NNsbWmjqaUt6YpERPrFngZET04tnggsz3q+Io7bjZlNBaYDD+aZPsfM5prZ3HXrEjgn4YDzGLdjORN3LOac6x/u//WLiCQgb0Dkucx3o5ltIZwT0Z1crYx8wXIxcK+7t+ea6O43u/tsd59dX5/AEbb7vR0H5lT9ifKU6dIbIlIS8gaEu9e6e12Oodbde3L96xXA5KznkwiXDc/lYoqxeymjph7b9xxOsadYum4zDyxYm3RFIiIFV8hjN58EZpnZdDOrJITAfZ1nMrN9CZcN/0cBa3njjnw/o7yBiyse5Rt/foV0Wq0IERncChYQ7t5GuNrrH4EFwN3uPt/MrjGzc7NmvQS4y4u932bmaVC7F2+vnMuCVY3c/8KqpCsSESmogt4qzd3vB+7vNO5LnZ5fXcga+kxZOaTKOLp9HjMrNvLpe57lrQeOp6JMJ9CJyOCkT7fe+MAfsFQ5Vw79I82taW55ZHHSFYmIFIwCojeGT4Khozi/9XdMH7qNr/7hJV5v2J50VSIiBaGA6K0r7gd3fn3kcwCcff3DOuxVRAYlBURvjZkFw0ZTN+9G/uP08TRsa+XnTy7v/nUiIgOMAmJPXP47AC5ruYu6IeV8/pfPs2hdU8JFiYj0LQXEnhi7H1TXY0/czAPv34uUGedc/whbmluTrkxEpM8oIPbUhx8BK6P+52/nJ1ccRXNrO2/66l9oa08nXZmISJ9QQOypmnp4y39C8yaOb3mYL19wMJu3t3LcdQ/oLGsRGRQUEG/E0f8MlTXwizlccnAtHz9tFuuadnDsdQ/oyCYRGfAUEG9EWTlc8XtIt8INR/GJ02cxYfgQ1m5p4VN3P8uONnU3icjApYB4oyYcAid9Frauw248hr9fdSqTRg7ll0+v5P23PMH6ppakKxQR2SMKiL5w4megqhY2LMQ2LeGRz57KjDHV/GPRBo677gH+vnB90hWKiPSaAqIvlJXDhx8Nj79/PLQ08eCnT+b3H38zZSnj0h8+znHXPcD2HTnvhyQiUpQUEH1l5FR4z93Qug2+fQik0+w/oY6n/u8Z1NdUsWpzM4de8yf+NH+1dmCLyICggOhLe58OI6fD9g3w209AOs2wynKe/OLp/HzOsZSZMef2eRx+zZ95TWdei0iRU0D0tY89DXWT4akf7wwJgGNmjOa5q9/CF87en8bmVk77xl+Z9YX7Wb5xW8IFi4jkZgOtu2P27Nk+d+7cpMvomjt862BoXA414+CTL4b9FNH6phbedv3DrG4MRziNqankrjnHsvfY2qQqFpFBzszmufvsXr1GAVEg7vDtg2HzchgyAj7xHAwZvsssaxqbOe+GR3YGxfCh5XzjXYdxyn5jKUtZElWLyCClgChG1x8JGxdCxTD4P/+AkdN2m2VDUwvn3fgoKzdtxwED6mur+PbFh3HM9NEKCxF5wxQQxWrx3+D288Pjy38HU47NOVtre5oHFqzh8798gY1bd+wcP7a2im+9+zCOmT6Kct0DW0T2gAKimK1fCDedAG3NMGomfHQeWP6WwbYdbZx7w6MsWb+VtnjxPwPG1FbxlQsO5ujpo6gdUtFPxYvIQKeAKHbbNsL1h0NzAwwdCf/yBNSM7fZl23e0c+4Nj7A4KywAUgZlKWPq6Gp++k/HMK5uSCGrF5EBTAExEKTT8MTN8IfPAgYX/xT2O6fHL29ubeepZZv49D3PsmZzM+1Zb58RAmPiiKHccOkR7Dehlgp1SYkICoiBZe1L8INToXUrHHAenPkVqNur14vZ0Zbm/BsfZeG6JtrTTnvayX5HUwYpM8pSxvQxoaUxuqaq734PERkQFBADTdsOuGE2NCwFK4Mz/gOOngPlb+wD/PWG7bzvR4+zbOM22h3SnULDgFTKSBmYGVNGDuW/3nUok0YOY0xNJdbFvhERGZiKLiDM7EzgO0AZ8EN3/0qOeS4CrgYceNbdL+1qmYMqIDI2LoYfnALbN0FZFbzje3DgBZDqu+6hC773KAtWNZJ2SHtsaTjkevczwZH9c2Z9DT/+wNGMrlaAiAxERRUQZlYGvAKcAawAngQucfcXs+aZBdwNnOrum8xsrLuv7Wq5gzIgMl57EO66LHQ7VdaE/RMzTi7oKre2tLFi03Y+eudTvLomXB+qLGWk3enqzqmZrivLhAjhoKy9x9ZwxwePpW5ouYJEpIgUW0AcB1zt7m+Nzz8H4O7XZc3zNeAVd/9hT5c7qAMCIN0Oz90Nv/k4tLdA1XC48BbY+7QuD4stlC3NrVz6g8dYuLaJtLPzLnmplOHdhAh0tEYMdguTltY0Q6vKmPfFMxhSUVbw30WklBVbQFwInOnu/xSfvxc4xt2vzJrnV4RWxgmEbqir3f0PXS130AdERmszfO842LQYcKgYGnZkH/Lu8LiINLe28+7//gevrG3CPXRddQ4SJ1x9pCs7QwR2tj7aYwJVlqewONO00dV8/V2HUjekgrqh5dRUlesEQpFuFFtAvAt4a6eAONrdP5o1z2+BVuAiYBLwMHCQuzd0WtYcYA7AlClTjly6dGlBai5KbTtg/i/gt58KXU+pcnjzv8JR/9SjcyiKzbYdbWxo2sE/3z6Pl1Y14kBFeWpnsAC7BEpv/zpTMWDIbrXE560xtKoqUjvHzRxTzXcuOZwhFWUMjUNVeYqULm8ig0yxBURPuphuAh5z99vi8weAq9z9yXzLLZkWRGfusOQRuPt9sH1jGFc9Fi65EyYemUj3U39pTztNLW1saW7lw7fP47X1W3fedCkTIpkP/7KUxXDpWaulO5lWS2brZlo26diyKS+zrPmyHtPRitoZSHGencsCZo2t4eb3zaayPEVleYqKshTlKdP+G+lzxRYQ5YTuo9OAlYSd1Je6+/ysec4k7Lh+v5mNAZ4GDnP3DfmWW7IBkW3Da/DY92Huj8DTUD4UTvzX0P00YkrS1RUdd2d7aztNLW186MdzeXVtU2iZeNbhvzFoMh/qmQ/+OFtmlp2vyex7MXrfyumNTBWZdWQaNtmBlHnelvadJ0vuHN8pZ9rimZUVZbZboGUvqyVuh73H1vDZM/ejrMyoSKUoSxkVZUZ5DLLyMqM81elxZt4yC+NT4TwchV6yiiogAMzsbODbhP0Lt7j7tWZ2DTDX3e+z8BfzDeBMoB241t3v6mqZCogszY3w4q/hj5+HlsYwrqoO3vKfsP+5MGxUsvWVmHTa2dGepqU1TUt7O3Oyw4iOgNn5mLCjHsI+lo6w2vW8FQfa28O4nR/+neaBrNDqnCoUNsTeiM6lWq6J8WHm9+vc+5fdOsvssypL2W7Bl5E9D9mB20kmTLO/LOSbpyLfPFldmxXlu+8ny37VjrY0leUpxtUNYU1jM8Aul8/5wAnTOG7mGPYdv2f3jSm6gCgEBUQem5aGo58e/ga0bQ/jho6Cc74O+5wFlcOSrU+KSjrttLvT1u60pdO0tTut6TTt6TCutT08bm0P58y0ptN84ZfPs2T91p3L2BluFfGDL8c+o0yLLPMBmu/TJvvD2PPrg9UDAAAP30lEQVTM2Jb9wd5Z/BxrzxMivvuseQMk12uKxbTRw3jo307Zo9cqICT89a96Fp6/J1zzqT1eNry6Hs7+L5hxCgwdkWyNIiUgc+BF9j6x9C4HY4ATDhX3HC3CXZcVQn1oZdkeHxKugJBdpdth6aMhLJ76Scf4qjp40ydh1hkw7qBBvYNbRAIFhOTX3gYr58K9H4Qtr4ed2wBllWHn9qwzwlnbnW6LKiKDgwJCem7Lalj4v/DnL8G2rIPGqurgzZ+C6SfC+EOhrDy5GkWkzyggZM+0t8LyJ+AXc3ZtXVgZ7H06TH8zTHszjD8YUrokhshAtCcBoa+HAmUVMO0E+FQ8RWXLGljycBie/Tm8+seOeYeOghP/DaYeB+MOVgtDZBBTC0K61/h6OIt78d/g6TvY5QDAISPguH+BKcfCxNk6nFakSKmLSfrH5pWw7B9heOZn0LqtY1plLcy+HKYcB5OPherRiZUpIh0UEJKM7Ztg+ZOw7O/w5I86zuoGKB8C+54Vrhc16SiYcBhUDMm/LBEpCAWEFIfWZnj9aVj+GPz9u7BtI7t0S1XWwGGXdoTGqBk6F0OkwBQQUry2rAnnYSx/AlbOg6V/B28P01IVMOst4Sip8QeFk/dGTlNoiPQhHcUkxat2HOx3ThggnOW97mVY8QQsewzm/wpe/l3H/FYGU4+HCYd2DKP31mG2Iv1ILQgpHju2wtoFsPo5WP18uKbUynkd0y0V92NkhUb9fuEwXRHpkloQMrBVVsOk2WHIaG+F9a+EsFj1bDhqavnjnV5XAwdfCOMPCTvBxx1QdLdlFRmI1IKQgSedho2LYNUzHcGx+K+7zjP2wI5WxviDYOwBuj+GlDS1IKQ0pFIwZu8wHHxhGOcODctC91QmNJ6/B579Wcfryipg6glQvz/U7xu6p+r3VXCI5KGAkMHBDEZODcP+b+8Yv2U1rJkf9m2sXQAv/goWPbTra1MVMPmYGBpxGLMv1I7XkVRS0tTFJKUnnYbNy8O+jXUvhaOpXvwVtGzZfd7KWjjg3HCuxuiZ4UiqUTPC/hKRAUTnQYi8Ee7QtCaGxiuw/mV47p54ZniO/xMrg8PfA6NmhvAYNRNGTdcOcilKCgiRQmlpCjvGN74GG+Lw0m/C+N3Cw8IhudX1cPxHO1oeI6ZCeWUS1YsoIEQS0bw5BMbGRfHna/DS/bAjR5cVFq6Ae+i7O1oeo2fC8Mk6CVAKSkcxiSRhyHCYeEQYOtu2sSM4NiwM4fHKH+Dxm3aft2IozDgVRs+I+zpiy0M7yyUhCgiRQho2KgyTOn1xc4emtR2hsWEhPHUHvPL7jjv6ZbMUVA2Hg98ZuqpGTu34OXRk//wuUnLUxSRSbNJpaFwRQmPDa7BpCTQshdceDJcjyaWiGmaesnt4jJiiI64EUBeTyOCQSoUP9hFTYOapu0/f3hBOCmxYCpuWdvx8+f7crQ8sXI5kn7fsGhwjpoZ9H9pxLnkoIEQGmqEjwjDhkN2nucPWdVnBsST8fPHX8MIvyHm4LgZVtbDf2zq1PqZC7YQQWFKSCtrFZGZnAt8ByoAfuvtXOk2/HPgvYGUcdYO7/7CrZaqLSeQNSLeHe4xntz7m3hJ2pnuanAFSPiRcej3T6hg5FUZMCz+HjdYO9AGiqA5zNbMy4BXgDGAF8CRwibu/mDXP5cBsd7+yp8tVQIgUUFsLNCyHhiUhQB7+Rjh50NOhdZIrQMYekGPfR/xZVdvfv4HkUWz7II4GFrr7IgAzuws4D3ixy1eJSHLKqzouhAhw1Ad3nd6yJez/yLQ+Hr2+45Ilefd/VMPep+3e+hg+WfcnL3KFDIiJwPKs5yuAY3LM904zO5HQ2vikuy/vPIOZzQHmAEyZMqUApYpIj1TVwrgDwwBw7Ec6prmHrqpM66NhGTz2/bBPZMFv8gdIVS3s81YYPikOU8LPEZPVAklYIQMiV8dk5/bpb4A73b3FzD4M/BjY7bANd78ZuBlCF1NfFyoifcAMqkeHYeKRYdybPtExPZ2GptW7HnmV+fn8veTegU44B2TICDj4XSE0hk+OP6dA9RjtAymgQgbECmBy1vNJwOvZM7j7hqynPwC+WsB6RCRJqRTU7RWGqcftPj3dHk4e3LwCNi8L+0I2Lw8/lzwMT/x37uWWD4Upx3aER6YlUjcxDOrG2mOFDIgngVlmNp1wlNLFwKXZM5jZBHdfFZ+eCywoYD0iUsxSZVA3IQyTj9p9ujs0N+waHJvjsPCBeJ+PPIfxVlaHc0qGT4bhE7O6sybDsDE6lDePggWEu7eZ2ZXAHwmHud7i7vPN7BpgrrvfB3zMzM4F2oCNwOWFqkdEBjizcFmRoSNznwMC0NoMjStjK2RFfLw8PH71z9C2PffryofApKN2bYFkDyV6NroutSEipcMdtm/qCJDNK0KANK6EV/+U5/LtAAYVw2DGSR3dV5kWyPBJUDMOyor7vONiO8xVRKS4mHVcQDFfK6S9Fbasgs0rOwIkEyaLHoLWbfkWHnaoDx0Jh7w77m+ZEMKkdkIYBthlTRQQIiLZyio6roWVT3Nj7L5aGXaoN74O834M2zbA9o3w2I15XmghpIYMhwPPh9pMiOzV8biqrmiOzFIXk4hIX3MPN5LasioESeOqjscv/jpMy3dmOsTLu9fCvmfH8IgtkQmHhp3se6CoLrVRKAoIERk02lpicLwehszj5+4O+0o6Xx9r1Ez42FN7tCrtgxARGUjKq2DktDBkO/O6jsfp9nA2euPK0IroRwoIEZFilioLt52tHd//q+73NYqIyICggBARkZwUECIikpMCQkREclJAiIhITgoIERHJSQEhIiI5KSBERCSnAXepDTNbByzdw5ePAdb3YTn9YSDWDAOzbtXcP1Rz/+hc81R3r+/NAgZcQLwRZja3t9ciSdpArBkGZt2quX+o5v7RFzWri0lERHJSQIiISE6lFhA3J13AHhiINcPArFs19w/V3D/ecM0ltQ9CRER6rtRaECIi0kMKCBERyalkAsLMzjSzl81soZldlXQ9+ZjZEjN73syeMbO5cdwoM/uzmb0af45MuMZbzGytmb2QNS5njRZcH7f7c2Z2RBHVfLWZrYzb+hkzOztr2udizS+b2VsTqnmymf3FzBaY2Xwz+3gcX7Tbuouai31bDzGzJ8zs2Vj3f8Tx083s8bitf25mlXF8VXy+ME6fVkQ132Zmi7O29WFxfO//Ptx90A9AGfAaMAOoBJ4FDki6rjy1LgHGdBr3NeCq+Pgq4KsJ13gicATwQnc1AmcDvwcMOBZ4vIhqvhr4dI55D4h/I1XA9Pi3U5ZAzROAI+LjWuCVWFvRbusuai72bW1ATXxcATwet+HdwMVx/E3AR+Lj/wPcFB9fDPy8iGq+Dbgwx/y9/vsolRbE0cBCd1/k7juAu4DzEq6pN84Dfhwf/xh4R4K14O5/AzZ2Gp2vxvOAn3jwGDDCzCb0T6Ud8tScz3nAXe7e4u6LgYWEv6F+5e6r3P2p+HgLsACYSBFv6y5qzqdYtrW7e1N8WhEHB04F7o3jO2/rzHtwL3CamVk/lQt0WXM+vf77KJWAmAgsz3q+gq7/aJPkwJ/MbJ6ZzYnjxrn7Kgj/gMDYxKrLL1+Nxb7tr4zN7Vuyuu6KrubYhXE44VvigNjWnWqGIt/WZlZmZs8Aa4E/E1ozDe7elqO2nXXH6ZuB0f1b8e41u3tmW18bt/W3zKyqc81Rt9u6VAIiV7IX6/G9J7j7EcBZwL+Y2YlJF/QGFfO2/z4wEzgMWAV8I44vqprNrAb4H+AT7t7Y1aw5xiVSd46ai35bu3u7ux8GTCK0YvbPNVv8WRR1d67ZzA4CPgfsBxwFjAI+G2fvdc2lEhArgMlZzycBrydUS5fc/fX4cy3wS8If6ppMUzD+XJtchXnlq7Fot727r4n/YGngB3R0bRRNzWZWQfig/am7/yKOLuptnavmgbCtM9y9AXiI0E8/wszK46Ts2nbWHacPp+ddmH0uq+YzYzefu3sLcCtvYFuXSkA8CcyKRyRUEnYq3ZdwTbsxs2ozq808Bt4CvECo9f1xtvcDv06mwi7lq/E+4H3xCIpjgc2Z7pGkdep/PZ+wrSHUfHE8UmU6MAt4IoH6DPgRsMDdv5k1qWi3db6aB8C2rjezEfHxUOB0wv6TvwAXxtk6b+vMe3Ah8KDHPcH9JU/NL2V9eTDCPpPsbd27v4/+3vOe1EDYg/8KoV/xC0nXk6fGGYQjOp4F5mfqJPRtPgC8Gn+OSrjOOwndBK2EbyUfzFcjoVl7Y9zuzwOzi6jm22NNz8V/nglZ838h1vwycFZCNb+J0AXwHPBMHM4u5m3dRc3Fvq0PAZ6O9b0AfCmOn0EIrIXAPUBVHD8kPl8Yp88oopofjNv6BeAOOo506vXfhy61ISIiOZVKF5OIiPSSAkJERHJSQIiISE4KCBERyUkBISIiOSkgREQkJwWESA+Y2WGdLlF9rvXRZePN7BNmNqwvliXSl3QehEgPmNnlhBOLrizAspfEZa/vxWvK3L29r2sRyaYWhAwqZjbNws1qfhBvovKneBmCXPPONLM/xCvnPmxm+8Xx7zKzF+KNWP4WL89yDfDueAOWd5vZ5WZ2Q5z/NjP7voUb5Swys5PiFUsXmNltWev7vpnNtV1v7vIxYC/gL2b2lzjuEgs3jXrBzL6a9fomM7vGzB4HjjOzr5jZi/GqnV8vzBaVkpbEae0aNBRqAKYBbcBh8fndwGV55n0AmBUfH0O4ng6EyxBMjI9HxJ+XAzdkvXbnc8INWu4iXMrgPKAROJjwBWxeVi2ZS2KUES6sdkh8voR4kyhCWCwD6oFywmUT3hGnOXBRZlmES1NYdp0aNPTloBaEDEaL3f2Z+HgeITR2ES9HfTxwT7ye/n8T7oYG8Chwm5l9iPBh3hO/cXcnhMsad3/ew5VL52et/yIze4pw/ZwDCXdT6+wo4CF3X+fhPgM/JdwND6CdcJVUCCHUDPzQzC4AtvWwTpEeK+9+FpEBpyXrcTuQq4spRbgZzGGdJ7j7h83sGOAcYOc9fXu4znSn9aeB8nil0k8DR7n7ptj1NCTHcrq6K1mzx/0O7t5mZkcDpxGuTnwl4e5nIn1GLQgpSR5uYrPYzN4FO2/ofmh8PNPdH3f3LwHrCdfQ30K4x/KeqgO2ApvNbBzhhlAZ2ct+HDjJzMaYWRlwCfDXzguLLaDh7n4/8AnCjXhE+pRaEFLK3gN838y+SLif712ES63/l5nNInybfyCOWwZcFbujruvtitz9WTN7mtDltIjQjZVxM/B7M1vl7qeY2ecI9yEw4H53z3X/j1rg12Y2JM73yd7WJNIdHeYqIiI5qYtJRERyUheTDHpmdiNwQqfR33H3W5OoR2SgUBeTiIjkpC4mERHJSQEhIiI5KSBERCQnBYSIiOT0/wFNWSCnHwvCTwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('2_nestimators.csv')\n",
    "        \n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(0, cvresult.shape[0])\n",
    "        \n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators5_1.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: from_csv is deprecated. Please use read_csv(...) instead. Note that some of the default arguments are different, so please refer to the documentation for from_csv when changing your function calls\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAANGCAYAAAB5u/b2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XmcXFWZ8PHfQ0ICARJ2QmQJCMgquwKDgCKiIO4zbrwjihvMqIzbC+iLEWXRYUAcEUZBUEbFFbeIgigqi+wCgbBKIJCQhaUTEpImyXn/uLfoolJdS3dV3+qq3/fzqU9u3XvuuU9VV7r76XPucyKlhCRJkiSpWGsUHYAkSZIkyeRMkiRJkjqCyZkkSZIkdQCTM0mSJEnqACZnkiRJktQBTM4kSZIkqQOYnEmSJElSBzA5kyRJkqQOYHImSZIkSR3A5EySykTEdyJieUTsVuXYiRGRIuKoiv0T82M3RsQzEfF8RMyLiN9FxHsiYnxZ26l5H+WPRRFxR0ScEBFjRuJ11hIRx0fEMUXHMRwRMSEipkXEIVWOHZO/71MLiGtKHtceI33tIkXEJRHxbNFxAOTvf4qIjYuORZIqmZxJ0oudADwBfDci1iztzJO1LwKXpJR+XbZ/e+B24HPAtcC/Aq8BPgY8DnwH+HyV6/w3sH/++BfgOuAc4Kutf0lNOx44pugghmkC8AXgkCrHppO973NHMqDcFLK4eio5kyQ1ZmzRAUhSJ0kpLYqIY4EryZKqL+RJ2qXAPLLkDYCIGAv8AtgQeEVKaWZFdz+OiFOBPatc6tGU0t/Knv8uInYF3g18qmUvSKtJKS0AFhQdRytFxISU0tKi45AkDY8jZ5JUIaX0B+AC4OSI2BuYBuwOHJtS6itr+lZgZ+C0KolZqa9HUkq/aPDSfcDz5TsiYo2I+GxE3JtPt5wfEd+LiC0qT46ID+TTI5dFxFMRcXlE7FTRZtuIuCwi5uT9zYuIq0vT7CJiFrALcHDZtMtZtYLO23wjIv5PRMyMiKV5HG9s8HWX9zUuIj5f9noXRMTFEbFJRbvXRMQ1EfFkRDwXEY9GxM/y6YxTGUi+vlD2Oi7Jz11tWmPe14yI2D8irs/7nBUR78+PHxkRt+Wv7a6IeH1FPNvlcT6Qt3k8In5dPj02n2J5c/704rK4ppW1eVNE3JD3sTgiroqI/SuuVZqWt1dE/DQingYeyo/V/PoO8p6fkPe3XZVjX4mI/tIUwIjYMyJ+k38Ol+fXmV7t8zgUEXFgHu/i/D24PiKOHKTdDfln/fGI+FJEfLDy6zrMWHaNiF9GxNP5df4eEe+raLNG/nm9L//MPBMRd0bEJ8rabBIR34qI2WWf6esi4rWtiFNSd3HkTJKq+wxwOPBTYEvggpTSVRVtDsv//dUQ+l8jspE3gEnAm4HXA1+paHc+8GHgG8BvgKnAl4BDImKvlNJCgIg4CTgd+CFwErARWVJ5Q0Tsm1J6IO/vt8AY4LPAo8DGwAHA+vnxt+avuY9seiPA8gZez5HAvsApwLN5/5dHxMtSSv9o4HwiYg3gl8CryKZ3Xg9sTTad9JqI2Cel9Fz+y/d04K/AB4BngJeQvX/jyKYrvh74HXARcGF+iXqjZZOBi/NrP0Y2NfU7EbEl8A6y97cvf42/iIhtU0pz8nOnAE8CJ+bX2RB4H3BjROyZUroPuA14f36NL+evgfxaRMR7gO+Tjdq+Gxifv4/XRMShKaVrK+L9OXAZ2R8S1sn31fv6VvO/ZJ+7YyibghvZ/Y9HA79OKS2MiHWAq4CHgX8jG0meDLwaWK9G/w2JiIPz/u8EjiX73B0P/Doi3p1S+lHe7uV5u/vJ3uOlwEfzWFsiIl5G9vmbD3yc7Gt7NHBJRGyWUipNP/4s2f+zLwN/AdYEduTF7/elwF5kU5/vz4/tRfZ/VJJeLKXkw4cPHz6qPMh+QU5kv+yvW+X4Ffnx8RX7g+yPX6XHmLJjU/Nzqj0urmi7Y77/vIr+X5HvPy1/vj7ZL6jTK9ptCSwDvp8/3yg/7xN1XvcM4Jom3qdEdp/eemX7NgNWAic20c+78r7eVrF/n3z/cfnzt+fPd6/R18Z5m2lVjh2TH5tatu+afN/eZfs2BFbk7+2Usv27520/VuP6Y8h+Ub8fOLvKazmmov0aZPco3gmsUbZ/XbIk6LqyfdPyPr5Y0UdDX99B4v0ZMLvi2m/I+3tj/nzv/Pmbh9D/JcCzddrckL/Wdcv2jQHuymOLfN+Pyf4AsHHF+3d35dd1kOuU3r+Na7T5Yf5/Z8uK/b8FlgCT8ue/Bm6vc73FwDnNvmc+fPjozYfTGiWpinwU52PAKmBTsl/IG/UJsumJpccdVdqcSzbStC/ZyMPJZIVBfljW5tX5v5eUn5hSugmYCRya79ofWLtKu9nAH8vaPUU2/e0zEfHJfIpaq34O/CmltLjs2vPIRh22bqKPN5KNgv06IsaWHsDfyZK/Q/J2fwf6gW9FxPsiYttWvABgbkrp1tKTlNJTZK/h72lghAyy9x7KXlse68kRcU9E9JMldf3A9sCLppYO4mVko2+XppRWlcXwLFnitF9ETKg452cVz4fz9b0Y2AIon2r3frL3/Yr8+YPA08BXIuKjEbFzg33XlY/KvRL4af6aAUgprSQbedqC7D0COBj4Y8pHjfN2q8iStlZ5DXB1/n+o3CVkxWZKU01vAnaPiG9GxOERMbFKXzcBx+TTH/eLskJDklTJ5EySqvs02S9g7wEeIJvetnZFm0fzfysTkB8wkHjdNkj/j6WUbskf16SUziCbrvjPEXF43qY07alaVcE5ZccbapdSSmSJ2u/JpmPdBiyIiK9HxHCnpT1ZZd9ysqSxUZuRjQL28+Lk9nmy6XMbA6SUHiJLIuYD5wEPRcRD5ff5DNFTVfb1V+5PKfXnm2uV7T6b7Ov3C+AoskRjX7LEvJH3oN7XcA1gg4r9L2o7zK/vFXl/pXvsNgDeBHwvT5BI2f2WB5Mlx6cDd+f3nH2xBQnHBmQjzoO9fnjx531elXbV9g3VRg3GcgbZ94r9yN7DJ/N75vYpO+edwHeBD5KNDj4V2X2jk1sYr6QuYXImSRXyEYFTyX4x/RHZNLjtgNMqmpbuQXtT+c6U0vxS4kU2palRd+b/lkbpSgnP5lXaTgEWNtmOlBUoOTalNJlsJOIcsvt6/rOJONtlIdlr2XeQR+keOFJKf00pHUV2v95+ZL/0fi0i3jXSQeeOJvu8nJxS+n1K6ab869/oWlr1voaryEatyqXKhkP9+paNUL0lItYn+6PEeLIRtfJ2d6WU3kWWnOwB/IjsHrzhVhh9muw1Dvb64cWf982qtGtlsvNkI7GklFaklM5OKe1FNg323WTTiX9fGulMKS1MKZ2QUppK9oeck4C3UTHSLUlgciZJL5JPo/su2S9fnwBIWcn7s4FPRMQ/lTW/HLiHrKrjji24fKmi3vz83z/m/76o0EFE7Es2Ve7qfNcNwHNV2m1BPj2r2sVSSvenlL5Mdk/PXmWHmh3xapXfkP3SP6ZsVLH8cV/lCSmllSmlG8kKVMDA6ygVMRmp15GoKJySVxl8SUW7weK6j+yes/dERJT1sQ7ZPXY3pCZL5df4+g7mYrLRwHeT/UHihpTSvYP0nVJKd6SU/oNsKmoj/deKdQlwI/C28hHqfFrm0WRFU+7Pd/8ZeE2ULSKdt/vn4cRQ4er8GlMq9v8r2T2If6s8IaX0TErpp2SjuRuS3V9a2ebRlNI3yP6wM6z3TFJ3slqjJL3YSWRFG96QUnqmbP//I5uu9p2I2COl9FxKaWVEvIVsGtlNEfFtssIST5NNz3sl2ShYtTL7W0XEfvn2OmRTKE8CHiGrwkdK6b6I+BbwsYhYRTZtairZ9LnZZKMipJSeiYgvAadHxPfI7lvbiGyx42Vk1Q5LVe6+AfyEbKpmP1ny9nLgzLLY7gLeFRHvBP4BLEsp3dXMmzhElwHvBX4bEeeS3avzPNn9Rq8GfplSujwiPprHPZ1saulaZFUbAf4AkFJaHBGPAG+OiKvJpiYuTCnNalPsvyG7r+heshHQvckqfj5W0e4hskT6vRExk6ywxZyU0pyI+CxZtcbfRMT/kI1cfYbss3RivQCa+PpWlVK6NyJuIPscbklWJbS8/zeSjcL9guxzEWQjQOszMIpcy5iIeEeV/UtSSlfk170K+FNEnJXHfzywK/DufNomZCPYRwFXR8RpZO/nRxmoWLmKxhwVEauNbOcJ1hfJ7oH8U2RrFT5F9tk8EvhsPsWTiPg1WQGdW8iqdG5NthbiI8ADETEJ+BPZVOd7yUbS9yWrJvrzBuOU1EuKrkjiw4cPH53yIEuk+oFvDXJ8P7IKhGdX7J9I9ovlTQysVTaPrCT68cCEsrZTWb1K43NkIyfnAJMr+l6D7P6h+/LYFpAXSKgS37Fk9zgtJxvN+AWwc9nxTclGR0pJweK8/Qm8uErk1mQJ56I8vll13rcEfKPK/lnAJU1+DcaSTZH7e/6+LM7jvQDYruzr8PO8/2Vko5zXAEdV9HUo2X1Xy/IYL8n3H0P1ao0zBnkNv6n3mskSlAvzr/sSsjL/B+b9XlNx7rvy19RPRUVJsiUV/pa/9mfJks0DKs6fRpVqg41+feu8/x/K+14KTKw49jKyJOPB/PgzZKNd72ug30sYvErprLJ2B5KNWj2bX+MG8mqRFf0dmL9Py8juDfsq2f+TRF5JsUYs02rEksra7Uq2TMYzZP+n/s7qVTY/CVxH9v9yOVlSdiGwdX58PNlyGHeQfW9YSpakTaPs+4IPHz58lB6lsrSSJEmjVkRcSZZw71B0LJI0VE5rlCRJo0pEnA3cTja9d0OyKYeHkY0eS9KoZXImSWq7vNBKLatS2fpeUh1jyCqqTiabjngP8H9SSv9baFSSNExOa5QktVVETAUertPsiymlaW0PRpKkDubImSSp3eaQVair10aSpJ7myJkkSZIkdQAXoZYkSZKkDuC0xiGKiACmkK0jI0mSJKm3rQfMScOYmmhyNnRTgMeKDkKSJElSx9gCeHyoJ5ucDd1igNmzZzNx4sSiY5EkSZJUkEWLFrHlllvCMGfVmZwN08SJE03OJEmSJA2bBUEkSZIkqQOYnEmSJElSBzA5kyRJkqQOYHImSZIkSR3A5EySJEmSOoDJmSRJkiR1gMKTs4g4PiIejohlEXFrRLyqTvv1I+K8iJibnzMzIo4oOz4rIlKVx3llba6pcvyydr5OSZIkSaql0HXOIuKdwNeA44HrgI8AV0TEzimlR6u0HwdcBcwH3gE8BmzJixd72xcYU/Z81/ycn1R0923glLLnzw3rxUiSJEnSMBS9CPUngYtSShfmz0+IiMOB44CTqrT/ALAhcEBK6fl83yPlDVJKC8qfR8SJwEPAnyv6WppSemKY8UuSJElSSxQ2rTEfBdsbuLLi0JXAAYOc9ibgBuC8iJgXETMi4uSIGFOtcX6No4HvpJRSxeH3RsTCiLg7Is6KiPXqxDs+IiaWHkDN9pIkSZLUjCJHzjYmm344r2L/PGDyIOdsC7wG+D5wBLA9cB7Z6zi1Svu3AOsDl1Ts/z7wMPAE2bTHM4DdgcNqxHsS8IUaxyVJkiRpyIqe1ghQOaIVVfaVrEF2v9mHU0orgVsjYgrwGaonZ8cCV6SU5rzogil9u+zpjIh4ALglIvZKKd02yLXPAM4ue74e2T1vkiRJkjRsRSZnC4GVrD5Ktimrj6aVzAWezxOzkpnA5IgYl1LqL+2MiK2B1wJvayCW24DnyUbiqiZnKaXlwPKy/hvoVpIkSZIaU9g9Z3kidSurTyU8DLh+kNOuA7aLiPK4dwDmlidmufeTjbJNbyCcXYA1yZI/SZIkSRpxRa9zdjbwwYj4QETsFBHnAFsBFwBExPci4oyy9ucDGwHnRsQOEXEkcDLZfWcvyJO39wPfTSmtqDj20og4JSL2iYip+RppPwFuJ0v+JEmSJGnEFXrPWUrpRxGxEdl6Y5sDM4AjUkql8vhbAavK2s+OiNcB5wB3Ao8D5wJfqej6tfm536ly2X7gUOATwLrAbLLRtS9WTJeUJEmSpBETq1eYVyPycvp9fX19TJw4sehwJEmSJBVk0aJFTJo0CWBSSmnRUPspelqjJEmSJAmTM0mSJEnqCCZnkiRJktQBTM4kSZIkqQOYnEmSJElSBzA5kyRJkqQOYHImSZIkSR3A5EySJEmSOoDJmSRJkiR1AJMzSZIkSeoAJmeSJEmS1AFMziRJkiSpA5icSZIkSVIHMDmTJEmSpA5gcjbKLe1fwdQTpzP1xOks7V9RdDiSJEmShsjkTJIkSZI6gMmZJEmSJHUAkzNJkiRJ6gAmZ6Nc/4pVL2w/s7S/wEgkSZIkDYfJ2Sg3buzAl/DuOYsKjESSJEnScJicdRGTM0mSJGn0MjnrInfP6Ss6BEmSJElDZHLWRRw5kyRJkkYvk7MuMueZZTy1xKIgkiRJ0mhkcjbKTRg3lllnHsk2G68DwF2PO7VRkiRJGo1MzrrETpuvB8D7vnMTS/tXFByNJEmSpGaZnHWJXaZMKjoESZIkScNgctYldpkysegQJEmSJA2DyVmXKE1rBCwKIkmSJI1CJmddYr211nxh2/XOJEmSpNHH5KwLzXjc9c4kSZKk0cbkrAvdMfuZokOQJEmS1CSTsy50x2N9pJSKDkOSJElSE0zOulDfc88z68mlRYchSZIkqQkmZ13q9kefLjoESZIkSU0wOesSE8aNZdaZR/LBA7cB4PZHve9MkiRJGk1MzrrMnlttAMClf3uEqSdOZ2n/ioIjkiRJktQIk7Mus9fW6xcdgiRJkqQhMDnrMptPWpvNJo4vOgxJkiRJTTI560K7b+HomSRJkjTamJx1oZdvManoECRJkiQ1yeSsC+299QYvbLsYtSRJkjQ6mJx1oZ02n/jC9kMLlhQYiSRJkqRGmZx1oXFjB76stz7iYtSSJEnSaGBy1uVuMTmTJEmSRgWTsy43/c65LkYtSZIkjQImZ11owrixzDz19YxdI4oORZIkSVKDTM661NrjxrDLlIn1G0qSJEnqCCZnXay8pL4kSZKkzmZy1sX2mWpyJkmSJI0WJmddbM+tBpKzhc8uLzASSZIkSfWYnHWxSWuv+cK2651JkiRJnc3krEfcMsvkTJIkSepkJmc9wpEzSZIkqbOZnPWI++Ytpu+554sOQ5IkSdIgTM662IRxY5l15pFsteEEUoLdv3glS/tXFB2WJEmSpCpMznqAJfUlSZKkzmdyNtr1L4Fpk7JH/5KqTVyMWpIkSep8Jmc9YJ+y5Oy5/pUFRiJJkiRpMCZnPWCLDdZ+Yfv22c8UGIkkSZKkwZic9YCIeGH7hocWFhiJJEmSpMGYnI12K8vK4y95sm7z6x6s30aSJEnSyDM5G+3GrDmwvfC+us3vfWIxCxYvb2NAkiRJkobC5KybLLi3oWb7nvYH1zuTJEmSOozJWTdZUH3krLQY9QcP3GaEA5IkSZLUKJOzblJn5OyA7TZ6YTul1O5oJEmSJDXB5KybLLgPaiRde201sN7Zg/OfHYmIJEmSJDXI5Gy0G7cOfH4BrDEWli+CRY8P3nTswJf7Wqs2SpIkSR3F5KwbjB0HG22Xbc+f2dApf7l/QRsDkiRJktQsk7NuselO2b/z72mo+a2PPM3iZc/XbyhJkiRpRJicdYuNts/+veoU6F9St/mKVYndpl3J1BOnW1ZfkiRJ6gAmZ91ik5cVHYEkSZKkYTA56xab7DiwvWplcXFIkiRJGhKTs26x/lYD20/PqtqktBj1fV9+PWuPGzMycUmSJElqiMlZt1ijLNlacF/NpuPHjuGAl25Us40kSZKkkWVy1o0aqNh48A6bjEAgkiRJkhpVeHIWEcdHxMMRsSwibo2IV9Vpv35EnBcRc/NzZkbEEWXHp0VEqng8UdFH5O3mRMRzEXFNROzSrtc44ppMzuYtWtbOaCRJkiQ1oNDkLCLeCXwNOA3YE/grcEVEbDVI+3HAVcBU4B3Ay4APAY9XNL0b2LzssVvF8c8CnwT+HdgXeAK4KiLWG/aL6gQNJGebrDf+he2rZ85vZzSSJEmSGlD0yNkngYtSShemlGamlE4AZgPHDdL+A8CGwFtSStellB5JKV2bUrqjot2KlNITZY8FpQMREcAJwGkppZ+nlGYA7wMmAO9p9QssxDOPwrK+hpt/efpM1zuTJEmSClZYcpaPgu0NXFlx6ErggEFOexNwA3BeRMyLiBkRcXJEVJYe3D6fsvhwRFwWEduWHdsGmFx+3ZTScuDPNa5LRIyPiImlB9DZo2zz7i46AkmSJElNKHLkbGNgDDCvYv88suSpmm3JpjOOAY4Avgx8CvhcWZsbgX8FDieb8jgZuD4iSuUJS303c12Ak4C+ssdjNdoW74kZRUcgSZIkqQlFT2sESBXPo8q+kjWA+cCHU0q3ppQuI7tf7YVpkCmlK1JKP0sp3ZVS+gNwZH7ofcO4LsAZwKSyxxY12o68cevAtD446DPZ8ys+A9MmQf+Sqs1La54dd/C2VY9LkiRJGllFJmcLgZWsPlq1KauPapXMBe5PKa0s2zcTmJxPk1xNSmkJcBewfb6rVLmxmeuSUlqeUlpUegCLB2tbqM12bar5oTtv9sL2kuXecyZJkiQVpbDkLKXUD9wKHFZx6DDg+kFOuw7YLiLK494BmJv3t5qIGA/sRJbYATxMlqAdVtZmHHBwjeuOHpMrC1PWttPkgVvnrNooSZIkFafoaY1nAx+MiA9ExE4RcQ6wFXABQER8LyLOKGt/PrARcG5E7BARRwInA+eVGkTEWRFxcERsExGvBH4KTAS+C5BSSmTl+0+OiLdGxK7AJcBS4Adtfr3tt8E2sOaEhptnxSszJ/78Lqs2SpIkSQUZW+TFU0o/ygt1nEK2HtkM4IiU0iN5k62AVWXtZ0fE64BzgDvJ1jc7F/hKWbdbAD8kKziyAPgbsF9ZnwBfBdYGvglsQFZE5HUppc6cqtiMNdaATXeCx28tOhJJkiRJTSg0OQNIKX2TLEmqduyQKvtuAPar0d+7GrhmAqblj+6z6S4mZ5IkSdIoU/S0RrXDZrs03LRUtfH/vXGnNgYkSZIkqR6Ts260+csHtlOt1QEGvH6XgeKVD85/ttURSZIkSarD5KwbbbLjwHbf7IZO2WCdgZUIfjfjiRotJUmSJLWDyVk3Gjt+YHvuHU2f/s1rHrJqoyRJkjTCTM663RCSM0mSJEkjz+Ss2829s+gIJEmSJDXA5KwbjVsHPnpttv3ItTBtEvQvabqb1GAxEUmSJEnDZ3LWrTbZCcau1XDzUkn9mz936Av77ps3+tfkliRJkkYLk7NuNWZsU+udlawzfmBd8stve5ypJ063OIgkSZI0AkzOutnmuw/r9N/cObdFgUiSJEmqx+Ssmw0zOXt66fMtCkSSJElSPSZn3WxyWXK2amVxcUiSJEmqy+Ssm2300oHtJx8sLg5JkiRJdZmcdbM1xgxsD2Ex6ldss2ELg5EkSZJUy9j6TdQV5v69oWalkvoAP7vtMW56+CkAdj7l9wDcc+rhTBjnx0aSJElqNUfOesXcO5s+5dAdN21DIJIkSZKqMTnrFfPuhpXNVV8cN9aPhyRJkjRS/O27V6xcDl/aGPqXFB2JJEmSpCpMzrrZuHVgWh9MPbDoSCRJkiTVYXLWC4a5GHW5nU/5PVNPnM7S/hUt61OSJEmSyVlvmNy65EySJElSe5ic9YIpewxsr1jW8Gmlsvr3nHp4G4KSJEmSVM7krBdMfMnA9ry7i4tDkiRJ0qBMznpBxMD27JuKi0OSJEnSoEzOes3sG1vSjYVBJEmSpNYyOes1j90Mq1YVHYUkSZKkCiZnvea5p+HUDZpajNrCIJIkSVL7mZxJkiRJUgcwOdOwLO1fwdQTp3v/mSRJkjRMJme9YNw6MK0P3v3Dlne9YPHylvcpSZIk9SKTs17ykn0Gthc93pIuv3/joy3pR5IkSep1Jme9ZNw6A9stKqn/k1sea0k/kiRJUq8zOetVj/yt6VOqVW1cvsKy/JIkSVIrmJz1qkevb3mXLkwtSZIkDZ3JWa966h+waM6wuzngpRu1IBhJkiRJJme97OydYNqkphakrnTcIdu2MCBJkiSpd5mcaVh22nxi0SFIkiRJXWFs0QFo9CkVBgG8v0ySJElqEUfOeklpMeqTHoMY07bLWBhEkiRJap7JWS8avx5svnvRUUiSJEkqY3LWq7Y+oOgIJEmSJJUxOetV5clZSsXFIUmSJAkwOetdW+w7sN03e8jdlIqD3HPq4S0ISpIkSepdJme9atw6A9vf3G/Y651Vs7R/BVNPnG5xEEmSJKkBJmdqm3ufWFx0CJIkSdKoYXKmtrn4ullFhyBJkiSNGiZnaps/zpz/wrZrn0mSJEm1mZz1qtKC1Fvu17ZLWANSkiRJapzJWa/bev+WdGPVRkmSJGl4TM563db/NLDd4vXO/mm7jVranyRJktTNTM563Uv2Htief09Lu/7owduuts97zyRJkqTqTM563djxA9sXHdbS9c522nxiS/qRJEmSeoHJmSRJkiR1gLFFB6DuUioMAjh1UZIkSWqCI2cqhPeeSZIkSS9mciZJkiRJHcDkTKtLq4qOQJIkSeo5Jme9btw6MK0PPvvwwL55d7ekaxemliRJkhpncqZMeUn9WX8dscsu7V/B1BOne/+ZJEmSep7JmVb38MglZ5IkSZIyJmda3cN/buli1LXMnLuo7deQJEmSRgOTM7VdrXvPzrnqgRe2La8vSZKkXmZypkLd9ugzRYcgSZIkdQSTM0mSJEnqACZnypRK6v/7zQP7lvW1/bKT1l6z7deQJEmSRgOTM73YxJcMbD9yfUu7rnbv2YcP2ma1dt57JkmSpF5kcqbBPfSntl/in/fZou3XkCRJkkYDkzMN7sGrYNWqtl5ijYi29i9JkiSNFpFSKjqGUSkiJgJ9fX19TJw4sehwWqd/CZw+ZfX9J8/J7ktrsaX9K9j5lN/XbHPPqYczYdzYll9bkiRJaoVFixYxadIkgEkppSEv5OvImSRJkiR1AJMzSZIkSeoAJmcaXIwpOgIgm/o49cTpVnCUJEm9kpYIAAAgAElEQVRSVzM504uV1jub1gdbvrLtl6tWXl+SJEnqRSZnGtwOrys6AkmSJKlnmJxpcNuXJWfPPVNYGAsWL39h2wWqJUmS1K1MzjS4DaYObJ+zM0yblJXab4Na0xs//4sZbbmmJEmS1EkKT84i4viIeDgilkXErRHxqjrt14+I8yJibn7OzIg4ouz4SRFxc0Qsjoj5EfGLiHhZRR/XRESqeFzWrteo4bn1kdVH7RxBkyRJUrcpNDmLiHcCXwNOA/YE/gpcERFbDdJ+HHAVMBV4B/Ay4EPA42XNDgbOA/YDDgPGAldGROUKyt8GNi97fKQlL0qSJEmShmBswdf/JHBRSunC/PkJEXE4cBxwUpX2HwA2BA5IKT2f73ukvEFK6fXlzyPi/cB8YG/gL2WHlqaUnmg00IgYD4wv27Veo+dqeN68xxR++fc5VY/tfMrvAbjn1MOZMK7oj7MkSZI0dIWNnOWjYHsDV1YcuhI4YJDT3gTcAJwXEfMiYkZEnBxRc0GuSfm/T1Xsf29ELIyIuyPirIiol2ydBPSVPR6r0370K5XV3+O9I3bJaveefebwHUbs+pIkSVJRipzWuDEwBphXsX8eMHmQc7Ylm844BjgC+DLwKeBz1RpHRABnA9emlMqrSnwfeDdwCPAl4O3Az+vEewZZold6bFGnfffYoaxIR0ojfvm11uyMxbAlSZKkduqEeWCVv+1HlX0la5BNUfxwSmklcGtETAE+A5xapf03gJcDB77ogil9u+zpjIh4ALglIvZKKd1WNciUlgMv1HTP8r4esfU/DWzPuwu22r+4WCRJkqQuVeTI2UJgJauPkm3K6qNpJXOB+/PErGQmMDmfJvmCiPhvsmmQr04p1ZuCeBvwPLB9g7H3ljXXHti+73fFxVHD0v4VTD1xuhUcJUmSNGoVlpyllPqBW8kqKpY7DLh+kNOuA7aLiPK4dwDm5v0RmW8AbwNek1J6uIFwdgHWJEv+VMt1X2vremclpXvPZp15pIU+JEmS1BOKXufsbOCDEfGBiNgpIs4BtgIuAIiI70XEGWXtzwc2As6NiB0i4kjgZLLS+SXnAUcD7wEWR8Tk/LF23udLI+KUiNgnIqbma6T9BLidLPnTKOcaaJIkSRqNCh2SSCn9KCI2Ak4hW2tsBnBESqlUHn8rYFVZ+9kR8TrgHOBOsvXNzgW+Utbtcfm/11Rc7v3AJUA/cCjwCWBdYDYwHfhixXRJjSKpgEIlkiRJUiuFv9QOTURMBPr6+vqYOHFi0eG0V/8SOH3Ki/edPCcrtT+ClvaveGFds0qfft0OnHXl/VWPuQaaJEmS2mnRokVMmjQJYFJKadFQ+yl6WqNGg9J6Z8ffOLBvaeWyccU65w8PFB2CJEmSNCwmZ2rc+lsObN/fWVUbV64afATYe9AkSZI0GpicaWh+++kRqdpYrlTB8Z5TD1/t2FYbThixOCRJkqR2MDlTV/jqO3ar28a10CRJktTJTM7UFbbbdN2iQ5AkSZKGxeRMPcn70CRJktRpTM40PM/OH/FL1rr3TJIkSRqtTM7UuFJJ/ZPnDOy79zfFxdMCjqBJkiSpU7gyr4bnjh/BlZ/Ptkd4YerSCBrQUHK1YuWqdockSZIkDZkjZxqeeXcVHUHDvjz93kGPOYImSZKkopmcqWf85s65RYcgSZIkDcrkTF3BIiGSJEka7UzONDwTNio6gob9+6tfWreNC1VLkiSpKCZnGp6d31J0BC9SawTtmH+aOvIBSZIkSQ2yWqOaVyqpD/DYLXDLRdn2sr4RrdbYbjuf8nsA7jn1cCaM87+KJEmS2suRMw3PJjsObJ+9E0ybBP1LiotHkiRJGqVMzjQ8EUVHUFUrC4RYZl+SJEkjweRMKrNyVRr0mEmaJEmS2snkTCpz5hWDL1QtSZIktZNVDtTVStMbgYZGvC6/fU7dNkv7V1gsRJIkSS3nyJmGp1S58bgbBvY9O7+4eCRJkqRRyuRMrbHB1gPb9/yyuDhqaKRIyEcO2qapPr0PTZIkSa1icqbW+8MXRm1J/Q++qrnkTJIkSWoVkzP1nFojaDHEpQEcQZMkSdJwmZxJLWSSJkmSpKEyOVP7pFVFR9AWS5abeEmSJKn1TM7UGqWqjZ/9x8C+R64vLp4GNFIgpJoPfe/Wum2W9q9g6onTHUWTJElSw0zO1Fpj1xrYvu3SrDDIKC0OMpj75z3bVHunOkqSJKkRrp6r9nngd0VH0JBmF6reduN1+MfC7kk2JUmS1BkcOVP7rHy+6Aja4qJj9h7SeY6gSZIkqRaTM6lMI/ehrbfWmiMYkSRJknqFyZnaZ+z4oiPoSBYLkSRJUjUmZ2qtUtXGaX2w41ED+0+fMqoKgwy1kmOznOooSZKkEpMztc/e7ys6gsKcNn1m0SFIkiRplDE5U/tM2avoCIZtqCNol98+p6n2jqBJkiTJ5EztE1F0BIXZYMLQioaYpEmSJPUukzOpAaURtFlnHsmEcfWXB/zhh145rOtZNESSJKn3mJxJbbDxeoNXqlyxclVTfTmaJkmS1BtMztQ+pcqNH7ttYN8oq9pYzXArOX70f2+r36gKkzRJkqTuZnKm9ltvctERdJS/z+4rOgRJkiR1IJMzaYiGOoL20k3WGdZ1HUGTJEnqTiZn0gi75P37tqQfi4ZIkiR1l/pl5yTVVBpBW9q/gp1P+X3d9muPG9PyGCqve8+phzdUVVKSJEmdw9/e1H6lwiBP3AUXHJjtO31K9u/Jc7LjXaCUpAHDHsm674nFwzq/lKyZpEmSJI0eTmvUyNlw26IjGDXee+FNLenHqY+SJEmjh8mZ1AbDLbcfLY4HLCQiSZLU6UzOpA70gw+9om19m6RJkiR1JpMzFWvJwqIjaKuhjqBtv9l6bYpIkiRJncpKASrWrZfAtWdn211UHKRSK4uFlDy7bHj9WDREkiSpszhyppFTqtp48pyBfXf9pLh4Rrk3/vd1LenHoiGSJEmdwT+Xq1h9s4uOYMQ1uy7aYJ5d3vpEyvXSJEmSiuPImVSQ4VZ0PPNtu7Y4otVZPESSJGnkmJxJo9Rrd95sxK7l1EdJkqT2MzlTsTbbbWD79CkwbRL0LykungIMdwStlrOvur/lfTqaJkmS1B7eTKKRVyoMAnDzhTD9U8XG0yHaUdHxBze2754+qz1KkiS1liNnKtauby86go7UqtG03V4ysUURDc6RNEmSpNYwOVOxxowrOoKONtwk7TvH7NPiiAbnfWmSJEnDY3ImdbGIqNvmwfnPtvy6jqZJkiQ1z+RMnadHC4PU0s6iIe/61o0t77PEJE2SJKlx3sUvjSLtKBqyRsCq1JKuBmXxEEmSpPocOVOxSpUbT55TdCQ965f/dsCIXcv70iRJkgbnn7ClUao0ira0f8ULI1NDsfn6a9dt89mf3jnk/gdTGbOjapIkqdf5m5A0yrUqSavlj/cuaEu/5arFb8ImSZJ6idMa1bksDNIxjnr55oVc14IikiSpl5icSV2inRUdv/Cmneu2mfPMcy2/bkkpSVv47DLvWZMkSV3L+ULqDKXCIP1LshEzDVk7Kjo24q3n3TBi1wLvWZMkSd3HkTOpi7VzNK3SytTmevx1OAVSkiSNdiZn6nwrlhUdwag3Eknapcfu27a+m2G5fkmSNFqZnKnz3XZpVhjE4iDD1s4kbafNJ9Ztc9lNs1t+3VocTZMkSaOJN2ios1S79+zG84uNqQsVdV/aWVfeP2LXKle6P8370iRJUidz5Eydb/ETRUfQ1UbyvrRtN1mn7deoxaqPkiSpk5mcSQJGJkn70YdfOeixt58/stUeS5z6KEmSOoXze9T5Jm4Bix7LtktTHU+ek02BVMuVkrSl/StWK1c/XBEx6LFHnlza0ms1y9L8kiSpaI6cqfPt/29FR9CTSknarDOPHJEk5bS37lK3zUiObln1UZIkjTT/LKzOVCoMArCsD35/UrHx9Lh2jqaVHL7LZD53+d0127zmrL+05dr1OKomSZJGQkeMnEXE8RHxcEQsi4hbI+JVddqvHxHnRcTc/JyZEXFEM31GxPiI+O+IWBgRSyLiVxGxRTten4ZpDX8J7hQjWTykmhWril3ousT71CRJUjsU/ltvRLwT+BpwPHAd8BHgiojYOaX0aJX244CrgPnAO4DHgC2BxU32+TXgKOBdwJPAfwG/iYi9U0or2/FapW5RXoq/ZCQSlcuP35+3frN24ZCURi6BqzaS6KiaJEkaqk4YOfskcFFK6cKU0syU0gnAbOC4Qdp/ANgQeEtK6bqU0iMppWtTSnc02mdETAKOBT6VUvpDSul24GhgN+C1bXmVaq3Tp7godQ/acsMJddv8n4tuHoFIBueomiRJGqpC/7ybj4LtDZxZcehK4IBBTnsTcANwXkS8GVgA/AD4SkppZYN97g2sme8DIKU0JyJm5G1Wu6kmIsYD48t2rVf3BUo9pHI0rZ33p9Vy7xOL6zcaAaXXfsvnD2WfL18NOKomSZJqK/q3hI2BMcC8iv3zgMmDnLMt8Brg+8ARwPbAeWSv5dQG+5wM9KeUnm7iuicBX6jxWtQupeIg/UsGSumr441EEZFqjjtkW86/5h8jdr1mDPY+mLRJkiTojGmNAJU3iUSVfSVrkN1v9uGU0q0ppcuA01h9GmQzfTbS5gxgUtnD4iFSA0a6JP+xB25Tt837Ly526mMly/ZLkiQofuRsIbCS1UerNmX1ka+SucDzFUU7ZgKT8ymNjfT5BDAuIjaoGD3bFLi+2kVTSsuB5aXntRbT1QhyUepRpajRtEp3Pb6osGvXU/m+OC1SkqTeUejIWUqpH7gVOKzi0GEMkiSRVV/cLiLKY98BmJtS6m+wz1uB58vbRMTmwK41riupRYouyT/tTTvXbXPCZX8fgUiaY7ERSZK6Wyf8CfZs4NKIuIWs0MeHga2ACwAi4nvA4yml0irE5wMfA86NiP8mu+fsZODrjfaZUuqLiIuA/4qIJ4GngLOAu4A/tPG1aji896zrVBYRGalRtTe+fHOm/eqemm2uffDJtsYwHJbvlySpOxX+0zyl9KOI2Ag4BdgcmAEckVJ6JG+yFbCqrP3siHgdcA5wJ/A4cC7wlSb6BPgPYAXwY2Bt4GrgGNc4G6WWLXJaY5folKmPHzloG/7nLw/XbPPks8trHh8prrcmSVJ36Iif3CmlbwLfHOTYIVX23QDsN9Q+8+PLyEbgPtZMrOpQ130Nbrwg2/b+s65QdJL2oYO2rZucHf61a0comuZ575okSaOPP53VHW69pOgI1CblUx+916o1HGWTJKkz+dNYo0+1e89WLCs2Jo2ITlnouuSq/3gVh53z15ptTvr5XSMUzdC5YLYkSZ3Bn7ySRq2ipz5usM64um2uumd+3TYp1VuCceQN9n6awEmS1D7+VFV3eOlr4KE/ZtuufdZzOnnq43GHbMv51/yjZpsjvn7dCEXTWk6PlCSptQpd50waltL0xml98OrPFx2NOkTRa6hVOvbAbeq2WbB48KqP1z/UuSX9K5XWYVv47DKmnjjdNdkkSWqSf+JUd9h0x6IjUIcpespjM77xnj349x9UX/T64z/svMWwm+HomiRJjfMnpKSuVllEBDpv6uN+22406LHJE9fiiUXdU/DG4iOSJA3On4bqXt57pkGMplG1X3/sAPY97Y8127zn2zeOUDTt0UjxkUomdJKkbuRPNkk9q9qoWkmnjK5FRN029897tm6bTqwIORy1EmsTN0nSaOVPL3WHamufScPQaWuq1XLm23flxJ/NqNnmkLP+PELRFK/a1MnBmMhJkjqJP5HU/dKqoiNQF+jke9deu9NmQO3kbMnylXX7uX/e4hZFNHo0knR7f5wkaaT4E0bd74wtsn+990wtNpruXbvsw6/kXd+qfW/ae7590whFM3q5OLckqZ386SFJwzQapkBut+m6ddtMXGssi5bVHg183Tl/aVVIXauZAicmcpKkcv5EUHfx3jN1gMpkbbSMrl39qYPqVoZ8asnzdfu56p55rQqp61V+JhyBk6Te5nd99Y7nnnFaowozGkbXGqkM+b8ffAVHX1h7+uNJP699/5saY+ImSb3H7+zqHX/+Ktx2Sbbt/WcqWLXRtZJOTNxKdpy8Xt02u71kInc9vqhmm/P+9FCrQupJ3vsmSd3J79rqTtWmN95+abExSQ0qT9w6pSJkMy5+/751S9hffN2suv3c/PBTLYqoNw1lce8SEztJKobfedU7Uv1S4lKnGQ3TIYfiTbtvzq/umFuzzXHfv33QY5/56Z2tDkllGvmcmcBJUuv5XVW9Y8w4WNmfbZdG05zeqFFmNJXvr+WUo3aum5xtu8k6/GPBkqrH/nTvgrrX+PZf/jGk2NSYWp9BEzdJGhq/c6p37Psh+Nt5RUchtcRorQjZjB9/ZL9Bp9995vAd+M/f31/z/P/5y8N1r/GTWx4bUmyqrfQ5dPkASWqO3x3V3Ur3ngEsfdLkTF2tW6dAVvPOfbesm5y9dc8pXH77nJptvvK7++pe67d31R7hU3NqVaEcjAmdpF7hdzr1jrFrFR2BNKJ6YXStls8duVPd5OygHTbmL/cvrNnmlF/eU/daru3WXrU+t7WSO5M6SaON37HU27z3TD2ml0bXGnH2v+xed9TmFdtsyE11Kkc2srbbZTfPbio2DV+t6ZWVnIIpqRP4HUeSethoXW9tJH3zvXvW/cV+r63W57ZHn6nZ5qw60zABzr36gaZiU3s1MwXTRE5SK/hdRL2j2tpnkgZVmbiV9Nr0yEZ861/3rpvAvXanTfnDzPk121x6w6ODHvu/P7trSLFpZDQzSmciJ2kwfmeQwOmNUpMGS9xgdC6cPRLOfPtudX9pf/crtuSHN1Wf/nh1ncQO4JiLb67bZtaT1Zcn0Mhp5I8b5UmeyZzUO/yfLklqqcESN0fb6vvU63YYNDn71Ou257+urD3tccbji+pe4x3n/61um7eed33dNnOfea5uG7XGUIqhNHufnQmg1Bn8X6je4/RGqRDlSZuja8179yu2qpucnfXPL+fTP7mzZpv11hrL4mW13//ZT9dPvI76Rv0E7ue3PV63jTrDUBLA4bYxIZRW5/8IqdzCB2DKHkVHIXU9q0a2xyEv26Rumz99+uC6v0hf+K9788Hv3VqzzZpjgudXppptTv/tvXXjueS6WXXbqDsNZc27RpK8at9PTAQ1Wvgplcpd8X9hdj7lx/vPpBFTq/jIYCxM0j57bLV+3TbXnfhqXnHaH2u2edX2G/PXB2qvI/eNPz1U91ofvfS2QY9d/9CTdc9Xd6uW5A3WxoIt6nR+8tS7qk1vnF3/XgxJncPCJMVZI6Jum3PeWX8duSN3m8z0u56o2eaWR54e9NjHf/j3unGcf039BFAqabZgSzNtGk38ymMwWewtfqUlSV3JwiSjwxffvEvd5OxLb96F//fLu6se22Gzdbl/3rM1z7/o2ll14/iXC+r/ce43d8yt20aqZShTOduZLA71/PKEsTw+i8wMn++YVG69KbB4TrZteX2pK9VaeLvEBK6zvGG3yYMmZz/40Cvr/lL5jr1fwk9vrV2c5B8L6y8xMO3X99Rt865v3Vi3zQ1OxdQo18j3x5EsMtNNiWB3vAqpVV79OfjVvxUdhaSC1UrgTNxGnxPfsGPd5OyCo/fko/97e802+227IX/7x1M12zw4v/YoHsDHGpiK+YVBktFyjzy5dNBjq1LtYi2SOpPJmVS69wyy+89MziTVUKt4iYnb6LXP1A3rtvnGe/as+9f8896zJ//2g9pJ3tSNJjCrRmIF1J3qCfD2828Y9Fi9Yi0AJ/38rrptZjzeV7fNkuXe3ym1ismZVI/TGyU1qFaBEnAErhe8ctv6Sd5Pj9u/bpL38UO34+tXP1izzbrjx/LsMBKjq+6ZX7fNMRffUrfNwf/557ptPnBJ/X7unVt/EXWp25mcSZJUgGaXD7D6ZG/51/23rpucXfOZwdesu+o/XsVh5/y15vmfet32dRc2nzJpLeb0LasdbAPufKz+CNzRF91ct80bzr22bps/379g0GOPPVV7xFIqmsmZJEmjQL1RuZJ6bUzyesMG64yr2+bdr9iqbnL2q4/9U91Rvmv/7yEc+JVrarb5z3fsxmd+Wnsa5abrjWf+4uU12yyocxzgUz++c9Bjb/nm4FNBS06bPrNum2sfrL1+HzS2Bl8j9yg2MrVU3cPkTCpXbe2zEqc3SuoCrUryGmljItgb1lpzTN02r95x07ptfvuJA+smgv/7wVdw9IU31Wyz60smMuPx6lMkJ4wbw9L+lTXPv/z2ObUDBU647I66bRpZg6+R6p6NTC19VZ3kGGpPLb3wrw/XPf8XDbwvl97wSN02Nz9cu6gOwLV1Fq8H6Hvu+bptRiOTM0mS1BbNJILeg6dG7Dh5vbptLnn/voMmeX/57CF1E8APH7QN3/pL7WRlp83XY+bcxTXb7Dh5Pe59onabDSasydNLaycZjUwtfe752gkn1J5aesGf/1H3/C83MKJ4bp2puADHfb92wRyAE35UP/k99L/+UrfNaGRyJkmSCtdMMZVyjs6p1T580LZ1k7NLj31FQ6N89dpc9cmD6rZpZGrpL47fv+6Uza++Yzc+O8jU0rfuOaXuiOGrtt+Yv9YZ0XrDrpO5YkbtSqPbbrxO3XUFd958Pe6pk/x2K5MzqZpa0xufXQhfd4qjJHWCRkbnTODU7bbYcELdNq+pMbX0c0fuVDc5O+edu9dNEr/0ll3qJmc//uh+dfv5XgPJ7w0nvZr9z/hTzTajkcmZ1KxrTi86AklSEyymInWfNcesUXQIbWFyJjXrzsuKjkCSVICRLKZSauO9eFJvMTmTaqk1vVGSpDYb6r149Y6BI4FSJzI5k5o1eXd4Iq8iZHl9SdIo5Uig1HlMzqRmvfEcuPA1RUchSdKoM5yRwPI2JnnqViZnUiNK0xshm+IoSZIK087pns20KU8S7zn1cCaMG7tam1ZdayhtTGJHn0gpFR3DqBQRE4G+vr4+Jk6cWHQ4GknV7j/79IOw7ibFxCNJkqRCLVq0iEmTJgFMSiktGmo/3VmDUhppZ20H0yY5qiZJkqQhMzmTmlWa4nhy7cUaJUmSpGaYnEmSJElSBzA5k1rp9ClOb5QkSdKQmJxJQ+X0RkmSJLWQyZnUDkufzkbQHEWTJElSg0zOpHb446lFRyBJkqRRxuRMaoc7f1R0BJIkSRplXIR6iFyEWquptjh1uZPnZPepSZIkqau4CLXUybY7rOgIJEmSNMqYnEnt8Iavrr7PMvuSJEmqYWzRAUhdo1RaH0zAJEmS1DRHzqSR1r/UMvuSJElajcmZJEmSJHUAkzOpHUpTHE+es/qx+feMfDySJEnqeCZn0ki7/CNFRyBJkqQOZHImjbTFcwe2reAoSZKknMmZ1E7VpjeOd9FySZIkrc7kTBppR3199X2OoEmSJPU8kzNppE09sOgIJEmS1IGGvQh1REwEXgPcl1KaOfyQpC7U6ALVp0/J/j15TnaOJEmSekbTI2cR8eOI+Pd8e23gFuDHwJ0R8fYWxydJkiRJPWEo0xoPAv6ab78VCGB94OPA51sUlyRJkiT1lKEkZ5OAp/Lt1wM/SyktBaYD27cqMKlr1VqguqR/aVYgxCIhkiRJPWMoydlsYP+IWIcsObsy378BsKxVgUmSJElSLxlKQZCvAd8HngUeAa7J9x8E3NWasKQed//vio5AkiRJIyxSSs2fFLEPsCVwVUrp2XzfkcAzKaXrWhtiZ8qrVPb19fUxcaKLCmsY+pcMVGksGTseVix/8T4rOEqSJHWkRYsWMWnSJIBJKaVFQ+1nSOucpZRuSSldnlJ6NiLGRMQewPVDScwi4viIeDgilkXErRHxqhptj4mIVOWxVlmbWYO0Oa+szTVVjl/W9BshtUtlYiZJkqSuN5RS+l+LiGPz7THAn4HbgNkRcUiTfb2TbJrkacCeZFUgr4iIrWqctgjYvPyRUiq/123fiuOH5ft/UtHPtyvafaSZ2KW22nTnoiOQJEnSCBvKyNk7gDvy7aOAbYAdGUiymvFJ4KKU0oUppZkppRPICo4cV+OclFJ6ovxRcXBBxbE3Ag+RJZHlllb009dk7FJrVKve+LYLV293+hSrN0qSJHWxoSRnGwOlhOgI4CcppfuBi4DdGu0kIsYBezNQ7bHkSuCAGqeuGxGPRMRjEfGbiNizzjWOBr6TVr+57r0RsTAi7o6IsyJivTrxjo+IiaUHULO9NCzrblp0BJIkSRphQ6nWOA/YOSLmkpXSPz7fPwFY2UQ/GwNj8v4q+588yDn3AseQVYWcCHwCuC4idk8pPVCl/VvIFsi+pGL/94GHyZLMXYEzgN0ZmAJZzUnAF2ocl4anNIIGtUfHSsVDLBAiSZLUVYaSnF0M/BiYCyTgqnz/K8mSp2ZVjmhFlX1Zw5T+BvzthYYR15Hd7/Yx4ONVTjkWuCKl9KLVflNK3y57OiMiHgBuiYi9Ukq3DRLnGcDZZc/XAx4bpK0kSZIkNaXp5CylNC0iZpCV0v9JSqlUVm4lcGYTXS3Mz6kcJduU1UfTBotlVUTcDGxfeSwitgZeC7ytga5uA57P+6manOWv84USehHRSIhS+/QvdRRNkiSpiwxl5IyU0k+r7Ptuk330R8StZFMJLy87dBjwy0b6iCxD2oPqi1+/H5gPTG+gq12ANclGA6XilaY4VlsDTZIkSV1pSMlZRBwMfBrYiWwK4kzgP1NKf22yq7OBSyPiFuAG4MPAVsAF+XW+BzyeUjopf/4FsmmND5Ddc/ZxsuTs3yriW4MsOftuSmlFxbGXAu8Ffks2ercz8F/A7UBPLKCtLvH0rKIjkCRJUgs1nZxFxNFk9539HPg62T1iBwBXR8QxKaUfNNpXSulHEbERcArZWmMzgCNSSo/kTbYCVpWdsj7wLbKpkH1kCdVBKaWbKsg0VekAACAASURBVLp+bX7ud6pcth84lKyYyLpkpfunA19MKTVT0ERqv1ojaJe9e2Db6Y2SJEmjXqxeYb7OCREzgW+llM6p2P9J4EMppZ1aGF/Hysvp9/X19TFx4sSiw1G3a3R6o8mZJEnSiFu0aBGTJk0CmJRSWjTUfoayztm2wK+r7P8V2YLUklqt2kLVm+9RXDySJElquaEkZ7PJpgVWOjQ/Jmkk/HOVGjynT4Fpk2qvkyZJkqSONJSCIP8FfD0i9gCuJysIciDZ4tCfaF1okmpy+qIkSVJXGco6Z+dHxBPAp4B/yXfPBN6ZUmqoBL6kISpNb4Tao2MWCJEkSRp1hrrO2eW8eG0yImLNiNgqpfRoSyKTJEmSpB4ylHvOBrMz8HAL+5NUS7UiIZKk/9/enYfJdpX14v++JJwwJKfDHBACAhEQkEBkngQNCv5EQa4MIgQEuSBqRKZwMcQfAvdqDEFQuQwaQZAgAgoYDCCgJgFDwhQGGQyBmIG5O+SQdEjW/aOqOZUeqqvqdHft6v58nmc/p3rvtXetYlMn59trrXcDzKyJRs6AGfG9byR/aoojAMAs2MiRM6Br/unZ0+4BAAAjEs5g1g2b3vil9219fwAAmMjI0xqr6ifWaXK7fewLsJlUcAQA6LRx1px9Ir1nmtUqx5b2t43oFDCBpRG0xUv3BrEjnpyc9ZfT7RcAACMZJ5z96Kb1AtgcD3zuynBmBA0AoJNGDmettfM2syPABvGgagCAmaQgCAAAQAcIZwAAAB0gnMF2NqzM/pLFPclxc71t2DRIAAA2lXAGO923vjztHgAAEOEMeOvj975+6c2MoAEATEm1Nt6jyarq41n9eWYtyWVJvpTkpNbaB/e9e91VVbuTzM/Pz2f37t3T7g6MZvAZaKNQyREAYF0LCwuZm5tLkrnW2sKk15lk5Oy9SW6d5NIkH0zyoSTfS3KbJGcmuWmS91fVL07aKWAL3eCwafcAAIBMFs5umORPWmv3b639XmvtWa21ByQ5Psl1W2sPSfKHSX5/IzsKbIDVCoT8yhun1x8AAH5oknD2K0n+dpX9b+kfS//47SbtFLCFrnvDtY9ZgwYAsGX2n+Ccy5LcJ721ZYPu0z+W9ELf5fvQL2AzLY2gJYIXAEBHTBLOXpnk1VV1RHprzFqSeyR5SpKX9tv8bJKPb0gPgelb3LO3kIgiIQAAm2LscNZa+8OqOjfJM5P8Wn/3fyZ5amvtzf2fX53kLzamiwAAANvf2KX06VFKn21nWJn9+x6dnHbi1fcZQQMASLJxpfQnmdaYJOlPa7xDetMaP9taM40RtqvlwQwAgA03yUOob5xeZcafSvLdJJVkLr1nnj2mtfaNDe5jJxk5Y9vyoGoAgLFM8yHUr0yyO8kdW2vXb61dL8md+vv+dNKOAB324GOn3QMAgG1vknD2c0me3lr73NKO1tpnk/xmkoduVMeAKVntQdV3e8L0+gMAsENMEs6ukeSKVfZfMeH1gFnmQdUAABtikoIg/5LkFVX12NbaBUlSVT+S5OVJPrCRnQOmyIOqAQC21CQjXc9MclCSr1TVl6vqS0nO7e/77Y3sHDBDFvf0RtCMogEATGSSh1B/LcndqurIJLdPr1rjZ1tr79/ozgEz5PJLpt0DAICZtmEPoa6qWyT5g9bakzfkgh2nlD470rAy+ze/R3L+f1x9nzL7AMAOMM1S+mu5fpInbuD1gK5ZrZLjkuXBDACAsaiuCGyMa1xz5T6VHAEARiacAeNbbQTtof9nev0BANgGhDNgY9zh4WsfM4IGALCukas1VtXb12ly8D72BZg14z4LbXHP3oIiioUAAFzNOKX050c4/oZ96AsAAMCONXI4a609aTM7AuwAG/ToDgCA7ciaM2BjDCuzv+T9L9r72jo0AICrEc6ArfPJN0+7BwAAnVXNNKOJVNXuJPPz8/PZvXv3tLsD3bN46d7iHz9USdb4O0eBEABgRi0sLGRubi5J5lprC5Nex8gZsHUe+kfT7gEAQGcJZ8DmWG0N2h0fsXb7xT29NWjWoQEAO5RwBgAA0AHjPOcMYHyjPqj6K/++97UHVQMAO5CRM6Ab3v7UafcAAGCqhDOgG666YuU+z0IDAHYQ4QzYOsMeVH3nX9n6/gAAdIhwBnTDQ16y9jEjaADADiCcAVtvtRG0qun1BwCgA1RrBKZn1EqOSxb3qOQIAGxbRs4AAAA6QDgDumFYsZAln3/P3tfWoQEA24xwBsyOdx897R4AAGyaaq1Nuw8zqap2J5mfn5/P7t27p90d2H4WL927vmwU1qABAFOysLCQubm5JJlrrS1Meh0jZ8DsuNczpt0DAIBNI5wB3bTaGrT7PWt6/QEA2GTCGbA9KBACAMw4zzkDum3cZ6EBAMwoI2fA9rK4pzeCZhQNAJgxwhkwO0Z5FtrH37h1/QEA2EDCGbC9fOAP9r62Dg0AmCHCGTB7ho6g1ZZ3BwBgIwhnwOxaLaT90p+vbGcEDQCYAcIZsL3c9shp9wAAYCJK6QOzb9Ry+y+9We/PF1zQOwcAoEOMnAE7j3L7AEAHCWfA9jJKuf0rvr91/QEAGJFwBuw8b3ns3teKhQAAHSGcAdvTsBG0i8/Z+v4AAKxDOAN2nhv+2Mp9RtAAgCkTzoDtbbURtMe9dXr9AQBYw9TDWVU9o6rOrarLquqsqrr/kLZHVVVbZbvWQJvjVjl+0bLrVL/dBVX1/ar6UFXdcTM/JzBlSyHtuPnkwBuv3c4IGgAwJVMNZ1X16CQnJnlJkrsm+bckp1TVoUNOW0hy08GttXbZsjafWdbmzsuOPzfJs5I8M8ndk1yU5H1VddA+fSAAAIAJTXvk7FlJXt9ae11r7XOttaOTfC3J04ec01prFw1uq7T5wbI231g6UFWV5OgkL2mtvb21dk6SJya5TpLHrfWmVXVAVe1e2pIIcjCrRim3f9mCZ6EBAFtqauGsqnYlOSLJqcsOnZrkPkNOPbCqzquq86vq3VV111XaHNafsnhuVb2lqm49cOxHkxwy+L6ttcuTfHid9z0myfzAdv6QtsCse+ew3xEBAGy8aY6c3TDJfkkuXrb/4vTC02o+n+SoJA9P8tgklyU5raoOG2jz0SRPSPKzSZ7av9bpVXWD/vGla4/zvknysiRzA9vNh7QFZsGwEbT/+uDe19ahAQBbYP9pdyBJW/ZzrbKv17C1jyT5yA8bVp2W5Owkv5Xkt/ttThk45dNVdUaSL6c3dfGESd63f93Lk1w+8N5rNQW2g2sdnFz23avve+nNen++4IJesAMA2EDTHDn7ZpIrs3K06sZZOaq1qtbaVUnOTHLYkDaXJvn0QJulNWoTvy+wzSi3DwB0wNTCWWttMclZSY5cdujIJKePco1+cY/Dk1w4pM0BSe4w0Obc9ALakQNtdiV54KjvC+wA17/12sdMcwQANsG0pzWekOSNVfWxJGck+Y0khyZ5dZJU1RuS/Hdr7Zj+zy9Kb1rjF5PsTm8q4+FJfnPpglV1fJJ3JflqeqNhL+y3/eukV+qxqk5M8oKq+mL/Wi9IsifJmzf58wJdtjSClgheAMCWm2o4a62d3C/UcWx6zyM7J8nDWmvn9ZscmuSqgVMOTvKa9KYkzif5eJIHtNb+Y6DNzZP8bXoFR76RXpi718A1k+SPklw7yZ8nuV56RUQe0lq7ZGM/IbCtLe6xDg0A2DDV2po1MBii/6yz+fn5+ezevXva3QE2y+KlewPYcr/5seTPfrL3WjgDgB1rYWEhc3NzSTLXWluY9DrTfgg1wOx60yP3vrYODQDYR8IZwDDDnoX23a9ufX8AgG1LOAOY1C3uuXKfETQAYELCGcAoVhtBe9RJa7cX0gCAMQlnAJPa75rT7gEAsI1M+zlnALPFs9AAgE1i5AxgMy3u6U1vNMURAFiHcAYwqWGVHJd8+I/2vrYODQAYQjgD2FfDQtqZr9n6/gAAM0k4A9hM17zOyn1G0ACAVQhnAJvpcW+bdg8AgBkhnAFslNWmN97ox9ZubwQNABiglD7ARhu33P7inl5QS3rBbtd1N69vAEBnGTkDmLZLvzXtHgAAHVCttWn3YSZV1e4k8/Pz89m9e/e0uwN03eKle0fHlrv29ZPvf/vq+4ygAcDMWFhYyNzcXJLMtdYWJr2OkTOArTCs3P7yYAYA7EjCGcC03fPpK/cpFgIAO45wBrCVVhtBu//vTa8/AEBnqNYIMA2jVnRUyREAdgwjZwBd9t7nT7sHAMAWUa1xQqo1AhtuWEXHQUbQAKBTVGsE2AkOvMnKfYqFAMC2JJwBdMVqxUKe+O7p9QcA2FLCGUCXXft6ax8zggYA24pqjQBdM2olRwBgWzFyBtBlq011XO7yS3sjaEbRAGCmCWcAs+6tj9/72lRHAJhZSulPSCl9YCpGLbe/RNl9ANh0SukD0HPrB027BwDABhDOAGbJamvQHvGatdub5ggAM0O1RoBZpKIjAGw7Rs4AdgIVHQGg84QzgFk3Srn9tx21Zd0BACajWuOEVGsEOmvUio4qOQLAhlCtEYDVrTaSdshdVrZTLAQAOkU4A9gJHnfytHsAAKxDtUaA7WrUio5LUyBNcwSAqTJyBkDP4h4VHQFgioQzgJ1glIqOp79y72vr0QBgywlnADvJsJB2+iu2vj8AwA8JZwD01cpdRtAAYMsIZwA70WojaL/4Z2u3F9IAYNOp1giwky2v6PgPzxjefnGP6o4AsEmMnAEAAHSAcAZAzygVHT/zjr2vTXUEgA0lnAEwulOes3KfkAYAG0I4A2B017TGDAA2i3AGwNUNm974lPevfZ4RNADYJ8IZAKtbLaRd90brn7e4pxfSBDUAGItS+gAMt7zc/nquXNz7Wtl9ABiZkTMANtZf/8K0ewAAM6laa9Puw0yqqt1J5ufn57N79+5pdwdgay1eundUbBxG0ADYhhYWFjI3N5ckc621hUmvY+QMgPENKxpyl8eufZ6iIQCwJuEMgMmtFtKOfPH65ykaAgArCGcATJfRNABIolojABth3IqOF35qc/sDADNIQZAJKQgCsI5Ji4YsUTwEgBmhIAgAM6ym3QEA6BzhDIDNMayi4xPftf75ioYAsMMIZwBsvRvdfrz2ioYAsAMIZwBsrmEjaMNc+InN6Q8AdJSCIBNSEARgHygWAsA2oiAIALNr0tG0JdajAbANCWcAdNPj3zFaO+vRANgmhDMApmfYCNohd177vNWm5AtpAMw44QyA6Rt3muNbf21z+wMAU6AgyIQUBAHYZIqGADAjFAQBYOe65X3Xb6NoCAAzRjgDoJuGTXV81EnjXct6NABmgHAGQLetFtKqJruWkAZAhwlnAGxPJz9+2j0AgLEoCDIhBUEApmhfi4UsUTQEgA2gIAgAO9copffv+T/Xv46iIQB0iHAGwOwaFtLu/+yt7w8A7APhDIDZN+5DrJcMTu1XLASAKZt6OKuqZ1TVuVV1WVWdVVX3H9L2qKpqq2zXGmhzTFWdWVWXVNXXq+qdVXW7Zdf50CrXeMtmfk4AOuidI0x9BIAtsv8037yqHp3kxCTPSHJakqclOaWqfry19tU1TltIcrWw1Vq7bODHByb5syRnpvf5XpLk1P41B38d+tokxw78/P19+SwAdMDSCFoy2gjYlz+wct9SoRHFQgDYYlOt1lhVH01ydmvt6QP7Ppfkna21Y1Zpf1SSE1trB4/xHjdK8vUkD2yt/Wt/34eSfKK1dvQ+9F21RoBZMKyy4w0OS771xeHnC2kArGPmqzVW1a4kRyQ5ddmhU5PcZ8ipB1bVeVV1flW9u6ruus5bzfX//Pay/b9aVd+sqs9U1fFVddA6/T2gqnYvbUmGtgegI4atR/u1d65/voqOAGyRaU5rvGGS/ZJcvGz/xUkOWeOczyc5Ksmnk+xO8jtJTququ7TWVvzqs6oqyQlJ/r21ds7AoTclOTfJRUnulORlSe6S5Mgh/T0myYuGfyQAZsr+B4zX3pRHADbRVNec9S2fV1mr7Os1bO0jST7yw4ZVpyU5O8lvJfntVU55VZKfSHK/Zdd57cCP51TVF5N8rKru1lo7e41+viy9oLfkoCTnr9EWgK5ZGkEb9wHW311rCTQAbKxphrNvJrkyK0fJbpyVo2mraq1dVVVnJjls+bGqemWShyd5QGttvRB1dpIr+tdZNZy11i5PcvnA9UfpIgBdM27RkJMeunKfETQANsHU1py11haTnJWVUwmPTHL6KNfoT1s8PMmFg/uq6lVJHpnkwa21c0e41B2TXHPwOgDsAKM8H+0Hl699zLPRANhA037O2QlJnlJVT66qO1TVy5McmuTVSVJVb6iqly01rqoXVdXPVtWtq+rwJK9PL5y9euCaf5bk8Ukel+SSqjqkv127f43bVNWxVfWTVXWrqnpYkr9L8vH0yvkDwF4///L12ygaAsAGmGo4a62dnOTo9J439okkD0jysNbaef0mhya56cApByd5TZLPpVfV8UfSm7b4HwNtnp5ehcYPpTcStrQ9un98MclPJ/nnJP+Z5E/71/qZ1tqVG/sJAZgJw0bQ7vALW98fAHakqT7nbJZ5zhnANjZYNOTZX0qOv+3w9qu1sR4NYMfYqOecdaFaIwB0y7hFQ0594cp9ioYAMKZprzkDgNn3qbesfUzREABGJJwBwDCjVHQ8aIznpgHAGoQzABjFsJB21HvWP19FRwDWIZwBwDhWC2kHHDTeNUx1BGAVwhkAbIVvfXnlPiENgAGqNQLAJMat6HjSwza3PwDMPCNnALAV2pVrHzOCBkCEMwDYd6NUdHzM365/HUVDAHY04QwANsqwkHbzu299fwCYKcIZAGy0UUbSVnPl4t7XpjoC7DgKggDAZtmIoiEv7T/g+gUX9K4HwLZl5AwAuuI7X1n7mPVoANuecAYAW2GUqY73+I2t6w8AnSOcAcBWGhbSHvDctc+76gd7X1uPBrAtCWcAMAv+6qEr9wlpANuKcAYA0zBuRcfvnLv2MSENYFsQzgBgmpZC2nHzya7rrN3u3r+1dX0CYCqEMwDoimGjaff9nfXPV9ERYKYJZwDQNZM+xHqQqY4AM8dDqAGgq8Z9iPXfP2XlPg+xBpgZ1Vqbdh9mUlXtTjI/Pz+f3bt3T7s7AOwUi5fuDVyTENIANtzCwkLm5uaSZK61tjDpdUxrBIBZMmzK422PXP9869IAOks4A4Dt4pf+Yrz21qUBdIpwBgCzaNKiIfPnb05/ANhn1pxNyJozADpnlPVo19g/ueoHqx+zHg1gIhu15kw4m5BwBkBnKRoCsKUUBAEAxvfYk9dvo2gIwFQIZwCw3Qxbj/YjR4x3LUVDALaMcAYA29WkRUM+846V+4Q0gE23/7Q7AABssqWQlowWrk55zub2B4BVKQgyIQVBAJhpw4qGHHiT5HsXDz9f0RCAH1IQBACY3LApj0/94PrnKxoCsOGEMwDYyVYLafvtGu8a1qMBbAhrzgCA8delnfm6lfuWpkma8ggwEWvOJmTNGQDbnodZA4zEmjMAYHoOvMn6baxLAxiLcAYArG5Y0ZAnv2/r+wOwzVlzBgAMtxTSBqc57rrO+uf94PK9r5dPjzTlEWAF4QwAGM24RUPe+vjN7Q/ANiOcAQCb44KPr31saSTt2V9Kjr9t77XRNGCHs+YMABjfsPVoS274Y+Nd0/PSgB1OOAMAJjcspD3ubVvfH4AZJpwBAPtutZA2StGQ1RhBA3Yo4QwAmJ73Pn/tY0IasMMoCAIAbJxxKzqeM8LUx8U9ewuIKBoCbGNGzgCAzTFK0ZA7PHy8axpNA7Yx4QwAmJ6fP2Gy84Q0YBsSzgCAzTXKCNowp71i7WOLe3ohTVADtgHhDADYGpOGtDNeuTn9AegY4QwA6Lbr32a0dqY6AjNOOAMAttbSCNpx86M9C+2ofxrv+kIaMKOEMwBgekaZ6niN/dY+9o6nrX1MSANmjHAGAEzfpOvRvvyB9dsoGgLMCOEMAOiOcUPaLe873vWNpgEdJpwBAN0z6rq0R5002fWFNKCDhDMAoNuGjaZVbX1/ADaJcAYAzIZJ16W99/lrHzOCBnTI/tPuAADAWJZCWjJaqDrnbeu3WdzTC2pJL/ztuu7k/QOYkJEzAGB7u9X9x2tvNA2YEuEMAJhdo0x1fNRfbV1/APaBcAYAzL5J16Mt+fIHV+4zggZsMeEMAOAdT137mJAGbBEFQQCA7WOwWMiSUULVNfZPrvrB8DaKhgCbzMgZAMAT3zPtHgAIZwDANjfKerQb3Ga8a5rqCGwC4QwA2Bn2uWjIB1buE9KADSScAQA7y6Qh7R1PW/uYkAZsAAVBAICdabB4yCihav9rJT+4bHgbRUOAfWDkDABglNG0p6wyrXEYo2nAmIQzAIAlw0LagTeZ7JpCGjAi4QwAYLlJ16WdduLax4Q0YB3WnAEArGXcdWlnvGr9NtalAWswcgYAsFGu96PjtTeaBgwwcgYAMIrBUbSkF6iWRsCWPOmU5ITbj39tI2lAjJwBAGyca4zwe++vnrH2MSNpsKN1IpxV1TOq6tyquqyqzqqq+w9pe1RVtVW2a41zzao6oKpeWVXfrKpLq+ofq+rmm/UZAYBtZtKiIW/9tfXbLO7phTRBDXaUqYezqnp0khOTvCTJXZP8W5JTqurQIactJLnp4NZa++FTIUe85olJHpHkMUnul+TAJO+uqv026KMBADvBUkg7bj7ZdZ312+9/wOb3CZhJUw9nSZ6V5PWttde11j7XWjs6ydeSPH3IOa21dtHgNs41q2ouya8n+b3W2vtbax9P8vgkd07yMxv8+QCAnWKU0bSnfni8a5rqCDvGVMNZVe1KckSSU5cdOjXJfYacemBVnVdV51fVu6vqrmNe84gk1xxs01q7IMk5a71vfxrk7qUtyUHrfkAAYGcaFtKue8P1z9/zrZX7hDTY9qY9cnbDJPsluXjZ/ouTHLLGOZ9PclSShyd5bJLLkpxWVYeNcc1Dkiy21r4zxvsek2R+YDt/jXYAAD2Trkt79f3WPiakwbbVlVL6bdnPtcq+XsPWPpLkIz9sWHVakrOT/FaS357kmiO2eVmSEwZ+PigCGgAwinEfZn3VFeu38TBr2HamPXL2zSRXZuVo1Y2zcuRrVa21q5KcmWRp5GyUa16UZFdVXW/U922tXd5aW1jaklwySv8AAMb2hHeN195oGmwLUw1nrbXFJGclOXLZoSOTnD7KNaqqkhye5MIxrnlWkisG21TVTZPcadT3BQCYyChTHW98h/Wv09abEATMmi5MazwhyRur6mNJzkjyG0kOTfLqJKmqNyT579baMf2fX5TetMYvJtmd3lTGw5P85qjXbK3NV9Xrk/xJVX0rybeTHJ/k00nev6mfFgAg2RvSFi/dOz1xHG961Mp9pjnCTJt6OGutnVxVN0hybHrPLDsnycNaa+f1mxya5KqBUw5O8pr0pi3OJ/l4kge01v5jjGsmye8m+UGStya5dpIPJDmqtXblxn9KAIA1TBrSLvrk2scG16MtEdig86oZEp9Iv5z+/Pz8fHbv3j3t7gAA28koQe1uRyVnn7T6sWd/KTn+tlffJ5zBpllYWMjc3FySzPXrU0xk2gVBAABYbpR1aQ9+4drHvv7ZlfsUDYHOE84AALpq0uekveHhax8T0qCzhDMAgK4bN6TVCP/EW9zTC2mCGnSGcAYAMCuWQtpx88mu66zd7tffN951jaZBJwhnAACzaNho2sG3XP/8y1apWSCkwVQJZwAAs2zSdWmvuf/ax4Q0mIqpP+cMAIANsBTSktFC1Uht9niwNWwhI2cAANvNKKNpj3jteNc0mgabTjgDANiuhoW02zxosmsKabBphDMAgO1u0nVp7zt27WNCGmw4a84AAHaKcdelffLN67exLg02jJEzAICdaJTRtBvdfuv6AwhnAACs4dfeOV57Ux1hnwhnAAA72bARtGuMsALm3H9duU9Ig4kIZwAATF405O+fvPYxIQ3GoiAIAAB7jVs0ZNeByeL3hrdRNARGYuQMAIDVjTKa9j9P27r+wDYnnAEAMNywkDbKKFhre1+b6ghrEs4AABjNpOvSTn7cyn1CGqxQbfA3GYysqnYnmZ+fn8/u3bun3R0AgOlYvHTverJ9YS0aM2xhYSFzc3NJMtdaW5j0OgqCAAAwuaXRtGEh7W5HJWefNPw6g0VDlghs7DCmNQIAsO+GTXl88Au3vj8wg4QzAACm77zTV+6zLo0dRjgDAGDjTFo05O+esPYxIY0dwpozAAA23rgPs97/2skPvj+8jYdZs80ZOQMAYHONMpr2tH/buv5ARwlnAABsjWEh7doHr3/+JRftfW2qI9uQcAYAwNaadF3a6x60cp+QxjZizRkAANMx7rq0K69Y+5i1aGwDRs4AAJi+UUbTHv3m9a+zuKc3kmY0jRkknAEA0B3DQtot7rH++e2qva9NeWTGCGcAAHTPpOvS/uYRK/cJacwIa84AAOiucdelXfyZtY95ThodZ+QMAIDZMMpo2hFPGu1aRtPoIOEMAIDZMiykPeh/rX3eOW9fuU9Io0OEMwAAZtO469Le+9y1jwlpdIA1ZwAAzLZR16XN3SKZ/9rwa1mXxhQZOQMAYPsYNpr2lA+Mdy2jaWwx4QwAgJ2hRvin7zf+c+U+IY0tIpwBALD9TPqctL/++bWPCWlsMmvOAADYvsZ9TtoorEtjkxg5AwBgZxhlNO2J7xnvmkbT2EDCGQAAO8uwkHaj261//nfPW7lPSGMDCGcAAOxMk65Le93PrH1MSGMfWHMGAMDONva6tLZ+E+vSmICRMwAAWDLKaNqT3jveNY2mMSLhDAAAlhsW0m5w2/XPb6uMrglprEM4AwCAtUy6Lu1Nv7z2MSGNNVhzBgAA6xl3XdpFn1q/jXVpLGPkDAAAxjHKaNoRTx7vmkbTiHAGAACTGRbSHvSC9c+3Lo1lhDMAANgXSzcE+QAAEwZJREFUk65L+5tHrH1MSNuRhDMAAJiGi89Zv83inl5IE9R2BOEMAAA2wtII2nHzya7rrN/+Xs8Y7/pG07Y94QwAADbaKFMd7/esya4tpG1bwhkAAGyWSdejLfm349c+JqRtO55zBgAAm20ppC1euvfZZqP46KvXbzP4vLQlnps2k4QzAADYKuM+zPqmd0ku/OT47+Ph1jPJtEYAAJiGUaY8Pu5t+/Yepj7OFOEMAACmaVhIq1r//DNetfF9YiqEMwAA6IJJi4ecduL6bTwvbSZYcwYAAF0y7rq06986+fZ/jX5969E6y8gZAADMsqP+abLzrEfrHOEMAAC6apSpjtcYYTLcv5+w9jFTHjtDOAMAgK7b14dZf+TPR2tnNG2qhDMAAJgVk4a0g285XnshbSqEMwAAmDVLIe24+WTXddZv/6RT1j42rJiIkLalVGsEAIBZthTUFi/dW4lxuf12rX3+Xz5k/fdY3LPy2qo9bjgjZwAAsB3s67q0cRlV23DCGQAAbCfjhrQnn7pv7yekbRjTGgEAYDsa9WHW17/1+tf62kc3pk8MZeQMAAC2u32d8njyr67fxvPS9plwBgAAO8WkIe2aYxb+MNVxIsIZAADsNOOW4n/6aZO9j5A2FuEMAAB2slFG03YduP51Tnne2seWQtr3vmHq4xDCGQAAsO/r0j7z9xvbnx2oE+Gsqp5RVedW1WVVdVZV3X/E8x5TVa2q3rlsf1tje85Am6+scvx/b/RnAwCAmTJpSLv1g8Zrb8rjClMPZ1X16CQnJnlJkrsm+bckp1TVoeucd8skx/fbL3fTZduTk7Qky+P8scva/eHEHwQAALaTcdelPfK1k72PkPZDUw9nSZ6V5PWttde11j7XWjs6ydeSPH2tE6pqvyRvSvKiJP+1/Hhr7aLBLckvJvlga21520uWtf3ehn0qAADYLvZ1yuOSYSX5hbTpPoS6qnYlOSLJ8umEpya5z5BTj03yjdba69ebAllVN0ny80meuMrh51XV76cXBv8uyR+31hbXuM4BSQ4Y2HXQsPcFAIBtZymkLV7aC1PjGuVh1ot7Vl77BRf03nubm/bI2Q2T7Jfk4mX7L05yyGonVNV9k/x6kqeO+B5PTHJJkrcv2/+KJI9J8qAkr0pydJI/H3KdY5LMD2znj/j+AACwvUw6knaPp032fjtkVG2qI2cD2rKfa5V9qaqDkvxNkqe21r454rWfnORNrbXLrvaGrb184MdPVdV3krytqp7XWvvWKtd5WZITBn4+KAIaAAA72VJIS0YLTg94TvIf/3fy91saUdumI2nTDmffTHJlVo6S3TgrR9OS5DZJbpXkXVW1tO8aSVJVP0hyu9bal5cO9Kc83i7Jo0foy0f6f942yYpw1lq7PMnlA9ce4ZIAALBDDAa1ZPKpj2957PptBqc+bqOgNtVpjf31XWclOXLZoSOTnL7KKZ9Pcuckhw9s/5jkg/3XX1vW/teTnNVa++QI3blr/88LR+o8AACwtkmnPp5/5ub0ZwZMe+Qs6U0VfGNVfSzJGUl+I8mhSV6dJFX1hiT/3Vo7pj818ZzBk6vqu0nSWlu+f3eS/5Hk95a/YVXdO8m90gt180nunuTlSf6xtfbVDf10AACwk4079fG+Ryennbi5feqoqYez1trJVXWD7H3m2DlJHtZaO6/f5NAkV01w6cekt3btb1c5dnl6Ux1flF4FxvOSvDbJH03wPgAAwChGqfZ472euH86uunLj+9YB1dqKuhuMoD8yNz8/P5/du3dPuzsAADB7Vgtpz/5Scvxth5938KHJd/sT3jqw5mxhYSFzc3NJMtdaW5j0OtMupQ8AAOxUk65L++7ASqQ2ySS7bhLOAACA6VoKacfNJ7uus377Bx+793Vtn0izfT4JAAAw+0YZTbvbE7auP1tIOAMAALpn0imPM0w4AwAAumsHhbSpl9IHAABY17jPS5tBwhkAADBbBoPaNmJaIwAAQAcIZwAAAB0gnAEAAHSAcAYAANABwhkAAEAHCGcAAAAdIJwBAAB0gHAGAADQAcIZAABABwhnAAAAHSCcAQAAdIBwBgAA0AHCGQAAQAcIZwAAAB0gnAEAAHSAcAYAANABwhkAAEAHCGcAAAAdIJwBAAB0gHAGAADQAcIZAABABwhnAAAAHSCcAQAAdIBwBgAA0AHCGQAAQAcIZwAAAB0gnAEAAHSAcAYAANABwhkAAEAHCGcAAAAdIJwBAAB0gHAGAADQAcIZAABAB+w/7Q7MuoWFhWl3AQAAmKKNygTVWtuQC+00VfUjSc6fdj8AAIDOuHlr7b8nPVk4m1BVVZKbJblk2n3ZIQ5KLwzfPP43n0Xu3+xzD2efezj73MPZ5x7OvmH38KAkF7R9CFimNU6o/z/6xKmY8fSycJLkktaauaQzxv2bfe7h7HMPZ597OPvcw9m3zj3c53uqIAgAAEAHCGcAAAAdIJwxKy5P8gf9P5k97t/scw9nn3s4+9zD2ecezr5NvYcKggAAAHSAkTMAAIAOEM4AAAA6QDgDAADoAOEMAACgA4QzOqOqjquqtmy7aOB49dtcUFXfr6oPVdUdp9nnna6qHlBV7+rfk1ZVv7Ts+Lr3rKquV1VvrKr5/vbGqjp4az/JzjXCPTxple/lR5a1OaCqXllV36yqS6vqH6vq5lv7SXamqjqmqs6sqkuq6utV9c6qut2yNuven6o6tP//g0v77f60qnZt7afZmUa8hx9a5Xv4lmVt/F06JVX19Kr6VFUt9LczquqhA8d9BztuhHu4Zd9B4Yyu+UySmw5sdx449twkz0ryzCR3T3JRkvdV1UFb3Ul+6LpJPpnePVnNKPfszUkOT/Jz/e3wJG/crA6zwnr3MEnem6t/Lx+27PiJSR6R5DFJ7pfkwCTvrqr9Nry3LPfAJH+W5F5Jjkyyf5JTq+q6A22G3p/+n+9J7/8L9+u3++Ukf7JFn2GnG+UeJslrc/Xv4dOWHfd36fScn+T5SX6yv/1Lkn8Y+GWk72D3rXcPk636DrbWbLZObEmOS/KJNY5VkguTPG9g3wFJvpvkadPuu60lSUvyS+PcsyR36J93z4E29+rvu920P9NO25bfw/6+k5K8c8g5c0kWkzx6YN/NklyZ5Gen/Zl22pbkRv37+IBR70+Sh/Z/vtlAm8ckuSzJ7ml/pp22Lb+H/X0fSnLikHP8XdqxLcm3k/y67+Dsbkv3sP96y76DRs7omsP606vOraq3VNWt+/t/NMkhSU5dathauzzJh5PcZwr9ZH2j3LN7J5lvrX10oM1HkszHfe2Sn+pPt/pCVb22qm48cOyIJNfM1e/zBUnOiXs4DXP9P7/d/3OU+3PvJOf09y/55/R+mXLEpvaW1Sy/h0t+tT/d7TNVdfyyGQj+Lu2Iqtqvqh6T3ijYGfEdnDmr3MMlW/Id3H8f+g4b7aNJnpDkC0lukuSFSU7vDykf0m9z8bJzLk5yyy3rIeMY5Z4dkuTrq5z79YHzma5TkvxdkvPSC9wvTvIvVXVEP2wfkmSxtfadZeddHPdwS1VVJTkhyb+31s7p7x7l/hySZd/T1tp3qmox7uGWWuMeJsmbkpyb3tTwOyV5WZK7pDcNMvF36dRV1Z3T+4f8tZJ8L8kjWmufrarD4zs4E9a6h/3DW/YdFM7ojNbaKQM/frqqzkjy5SRPTLJUgKAtO61W2Ue3rHfPVrt/7mtHtNZOHvjxnKr6WHpB7eeTvH3Iqe7h1ntVkp9Ib83KenwPu2nVe9hae+3Aj+dU1ReTfKyq7tZaO3up2SrXcw+3zn+mt8bo4PTWi/11VT1wSHvfwe5Z9R621j67ld9B0xrprNbapUk+neSw9H5Tkaz87cONs3Jkhm4Y5Z5dlN4o6XI3ivvaSa21C9MLZ4f1d12UZFdVXW9ZU9/NLVRVr0zy8CQPaq2dP3BolPtzUZZ9T/vtrxn3cMsMuYerOTvJFbn699DfpVPUWltsrX2ptfax1tox6RVa+p34Ds6MIfdwNZv2HRTO6KyqOiC9BZYXZu9Q8pEDx3elV+Xq9Kl0kPWMcs/OSDJXVfcYaHPP9NZcuK8dVFU3SHKL9L6XSXJWev+BGrzPN01v2od7uMmq51VJHpnkwa21c5c1GeX+nJHkTv39Sx6S5PL++WyiEe7hau6Y3j/cl76H/i7tnkpvzZjv4Oxauoer2bTvYPWricDUVdXxSd6V5Kvp/Ubphen9Q/7OrbXzqup5SY5J8qQkX0zygiQ/lV4VnEum0ukdrqoOTHLb/o8fT69s/geTfLu19tVR7llVnZJe5aqlkrSvSXJea+0Xtupz7GTD7mF/Oy7J36f3H6BbJXlpkkOT3GHgHv5Fkv8vyVH9c45PcoMkR7TWrtyaT7IzVdWfJ3lckl9Mb0rOkvnW2vf7bYben34Z70+k99vd5yS5fvZW6fytrfkkO9d697CqbpPkV5P8U5JvJvnx9Eqsfz/J3Ze+Y/4unZ6qeml663O/luSg9CotPj/Jz7XW3uc72H3D7mGS/8pWfgenXabSZlvakrwlyQXplZz97/T+QfjjA8crvX8oXpheedkPJ7nTtPu9k7f0glZbZTtp1HuW3n+E/ibJQn/7myQHT/uz7ZRt2D1Mcu30KoZ9vf+9PK+//xbLrnGtJK9M8q0ke9L7JcstpvF5dtq2xr1rSY4a5/6kF7jf3T/+rX77A6b9+XbCtt49TG+k+sP9+3J5ki8leUWS6y+7jr9Lp3cPX5/kK/378/Uk709y5MBx38GOb8Pu4VZ/B42cAQAAdIA1ZwAAAB0gnAEAAHSAcAYAANABwhkAAEAHCGcAAAAdIJwBAAB0gHAGAADQAcIZAABABwhnALCOqvpKVR097X4AsL0JZwDQV1VHVdV3Vzl09ySv2YL3FwIBdrD9p90BAOi61to3pt2HcVTVrtba4rT7AcB4jJwB0DlV9aGq+tOq+qOq+nZVXVRVx4147lxVvaaqvl5VC1X1L1V1l4Hjd6mqD1bVJf3jZ1XVT1bVTyX5qyRzVdX623H9c642otU/9rSqendV7amqz1XVvavqtv2+X1pVZ1TVbQbOuU1V/UNVXVxV36uqM6vqZwY/c5JbJnn50vsPHPvlqvpMVV3e78vvLfvMX6mqF1bVSVU1n+S1VbWrql5VVRdW1WX9NseMdSMA2FLCGQBd9cQklya5Z5LnJjm2qo4cdkJVVZL3JDkkycOSHJHk7CQfqKrr95u9Kcn56U1VPCLJ/05yRZLTkxydZCHJTfvb8UPe7veTvCHJ4Uk+n+TNSf5vkpcl+cl+m1cNtD8wyT8l+Zkkd03yz0neVVWH9o8/st+vYwfeP1V1RJK3JnlLkjsnOS7Ji6vqqGX9eU6Sc/qf6cVJfjvJw5P8SpLbJXl8kq8M+TwATJlpjQB01adaa3/Qf/3Fqnpmkp9O8r4h5zwovQBz49ba5f19z66qX0ryqPTWjR2a5I9ba59fuvbSyf1Rp9Zau2iE/v1Va+2t/fP+T5Izkry4tfbP/X2vSG8kLuld9JNJPjlw/gur6hHpBahXtda+XVVXJrlk2fs/K8kHWmsv7v/8har68fTC2EkD7f6ltfbDMNkPfV9M8u+ttZbkvBE+EwBTZOQMgK761LKfL0xy43XOOSK9Eapv9acOfq+qvpfkR5MsTTE8Icnrqur9VfX8wamH+9C/i/t/fnrZvmtV1e4kqarr9qdpfraqvtvv1+3TC4vD3CHJacv2nZbksKrab2Dfx5a1OSm9Ub3/7E8Rfci6nwiAqRLOAOiqK5b93LL+f7eukV6IO3zZdrskf5wkrbXjktwxvemPD07y2f4I1r70rw3Zt9TnP07yy0n+V5L79/v16SS71nmfGrjW4L7lLh38obV2dnqh9PeTXDvJW6vqbeu8FwBTZFojANvJ2emtN/tBa+0razVqrX0hyRfSK77xt0melOQdSRaT7LfWefvo/klOaq29I0mq6sAkt1rWZrX3/2yS+y3bd58kX2itXTnsDVtrC0lOTnJyP5i9t6qu31r79mQfAYDNZOQMgO3k/emt/XpnVf1sVd2qqu5TVX/Yr8h47X4Fw5+qqltW1X3TKwzyuf75X0lyYFX9dFXdsKqus4F9+1KSR1bV4f3qkW/Oyv8OfyXJA6rqR6rqhv19f5Lkp6vq96vqx6rqiUmemeHFSlJVv1tVj6mq21fVjyX5H0kuSrLac9wA6ADhDIBto1/44mFJ/jXJX6Y3OvaW9EaoLk5yZZIbpFdl8QvpVUE8JcmL+uefnuTV6Y02fSO9KpEb5XeTfCe9qpDvSq9a49nL2hzb7+uX+++/ND3xV5I8Jr1qjP9/kmNbayet837fS/K89Naindm/7sNaa1ft8ycBYFNU779jAAAATJORMwAAgA4QzgCYGVX1q4Ml8pdtn5l2/wBgX5jWCMDMqKqDktxkjcNXtNY8aBmAmSWcAQAAdIBpjQAAAB0gnAEAAHSAcAYAANABwhkAAEAHCGcAAAAdIJwBAAB0gHAGAADQAf8PScFvkPjyRLUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('2_nestimators.csv')\n",
    "\n",
    "cvresult = cvresult.iloc[25:]\n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(25,cvresult.shape[0]+25)\n",
    "        \n",
    "fig = pyplot.figure(figsize=(10, 10), dpi=100)\n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators_detail.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "338\n"
     ]
    }
   ],
   "source": [
    "nestimators = pd.read_csv(\"2_nestimators.csv\")\n",
    "best_n_estimator = nestimators.shape[0]\n",
    "print(best_n_estimator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>n_estimators</th>\n",
       "      <th>test-mlogloss-mean</th>\n",
       "      <th>test-mlogloss-std</th>\n",
       "      <th>train-mlogloss-mean</th>\n",
       "      <th>train-mlogloss-std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>333</th>\n",
       "      <td>333</td>\n",
       "      <td>0.586504</td>\n",
       "      <td>0.002381</td>\n",
       "      <td>0.473750</td>\n",
       "      <td>0.001488</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334</th>\n",
       "      <td>334</td>\n",
       "      <td>0.586490</td>\n",
       "      <td>0.002335</td>\n",
       "      <td>0.473492</td>\n",
       "      <td>0.001514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>335</th>\n",
       "      <td>335</td>\n",
       "      <td>0.586525</td>\n",
       "      <td>0.002327</td>\n",
       "      <td>0.473237</td>\n",
       "      <td>0.001509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>336</th>\n",
       "      <td>336</td>\n",
       "      <td>0.586477</td>\n",
       "      <td>0.002349</td>\n",
       "      <td>0.472900</td>\n",
       "      <td>0.001531</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>337</th>\n",
       "      <td>337</td>\n",
       "      <td>0.586436</td>\n",
       "      <td>0.002342</td>\n",
       "      <td>0.472606</td>\n",
       "      <td>0.001573</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     n_estimators  test-mlogloss-mean  test-mlogloss-std  train-mlogloss-mean  \\\n",
       "333           333            0.586504           0.002381             0.473750   \n",
       "334           334            0.586490           0.002335             0.473492   \n",
       "335           335            0.586525           0.002327             0.473237   \n",
       "336           336            0.586477           0.002349             0.472900   \n",
       "337           337            0.586436           0.002342             0.472606   \n",
       "\n",
       "     train-mlogloss-std  \n",
       "333            0.001488  \n",
       "334            0.001514  \n",
       "335            0.001509  \n",
       "336            0.001531  \n",
       "337            0.001573  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_nestimators = pd.read_csv(\"2_nestimators.csv\")\n",
    "data_nestimators.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "338"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_3.get_params()['n_estimators']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在固定最佳的参数max_depth = 5,min_child_weight = 2，默认学习率learning_rate =0.1，重新调节n_estimator，最优的n_estimator = 276，比第一次调优的338小很多，而本轮的最优结果logloss = 0.587131 ，反而也比上一轮的0.583831大了一些，个人猜测：  \n",
    "1、经过以上几轮训练和参数调优，其实参数n_estimator,max_depth和min_child_weight的最优参数都缩小了，而且样本随机采样的比例是0.5，是否选取的样本太少了或这些参数取值太小了造成了当前的模型已经有点欠拟合了  \n",
    "2、默认的学习率learning_rate =0.1，是否学习率太大了，造成训练时在最优解附近震荡，收敛不到最优解\n",
    "\n",
    "尝试将学习率降低为0.09试一下（之前自学过深度学习，其中学习率是一个很重要的参数，一般训练时取值是随迭代次数增加，逐渐减小（收缩步长），这里先不考虑这么多）  \n",
    "结果logloss=0.586436 ，比learning_rate = 0.1时好一点，可能再降低学习率能获得更好的效果，在另一个文件中尝试了learning_rate =0.01，发现效果很差，由于计算太慢了，电脑跑不动，这里就暂时取最优的n_estimator = 338进行下一步的调优"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
